Selective Data Capture for Program Execution Coverage Expansion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current software testing methods face challenges in achieving comprehensive code execution coverage, particularly due to limitations in using artificial or minimal inputs, which often fail to reveal issues present in real-life production environments, and existing techniques struggle to identify and exercise all execution paths within a program.
Innovation Solution
The method involves identifying non-covered portions of a program and inserting triggers to capture execution coverage expansion data from real-life user interactions, which is then used to expand code execution coverage in a test environment, leveraging cloud infrastructure to gather and replicate real inputs and states from production environments.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If comprehensive testing of all possible inputs is performed, then detection of program deficiencies is improved, but testing time and resources become prohibitively large
Solution Approach 1:
The patent applies partial action by selectively capturing and testing only those inputs that lead to non-covered code portions, rather than exhaustively testing all possible inputs. The system identifies untested code regions and generates or selects test inputs specifically targeted at exercising those portions, achieving improved coverage without proportional increases in testing time.
Solution Approach 2:
The system employs self-service through automated feedback loops where execution results are analyzed to identify non-covered portions, which then automatically generate new test inputs. This self-directed process continues iteratively, with the testing system serving itself to expand coverage without requiring manual intervention for each test case generation.
2Difficulty of detecting and measuring
If static analysis tools are used to find program problems, then detection capability is improved, but not all deficiencies can be found
Solution Approach 1:
The patent merges static analysis with dynamic execution-based testing. While static analysis tools examine code without execution, the system combines this with actual program execution and input capture, creating a hybrid approach that leverages both the proactive code examination of static analysis and the realistic behavior observation of dynamic testing to achieve more complete deficiency detection.
Solution Approach 2:
Static analysis is performed as a preliminary action to identify potential problem areas before execution-based testing begins. This preliminary examination guides subsequent dynamic testing efforts, allowing the system to focus computational resources on areas most likely to contain deficiencies while maintaining comprehensive coverage.
3Extent of automation
If artificial test inputs are used for testing, then testing automation is improved, but representativeness of real-life scenarios deteriorates
Solution Approach 1:
The system uses captured real inputs from production environments as an intermediary between artificial test generation and actual program testing. Instead of directly using synthetic test data, the system captures actual inputs that occurred during real program execution, thereby mediating between the needs for automation and representativeness.
Solution Approach 2:
The patent employs copying by replicating real input data and execution states from production environments into the testing environment. These copied real-world inputs are then used to drive automated tests, preserving the authenticity and representativeness of actual usage scenarios while enabling systematic automated testing.
4Reliability
If triggers are inserted to capture execution coverage expansion data, then code execution coverage is improved, but device complexity increases
Solution Approach 1:
The system extracts only the essential elements needed for coverage expansion—specifically, the capture of inputs at critical points in the execution flow. Rather than implementing comprehensive monitoring of all program aspects, it selectively extracts and captures only those data elements necessary for identifying and exercising non-covered portions.
Solution Approach 2:
The patent applies parameter changes by dynamically adjusting which code portions are targeted for coverage based on execution feedback. The system modifies its testing parameters iteratively, focusing subsequent testing efforts on previously uncovered areas, thereby adapting the testing infrastructure to maximize coverage efficiency without proportional increases in complexity.
Data Source
Figure 1~3
Figure 4~5
Figure 6~7
AI summary
Program execution coverage is expanded by selectively capturing inputs and execution state information in a production environment after identifying test insufficiencies in a test environment. A program being tested has a non-covered portion and a covered portion. The non-covered portion includes code which is not exercised by a set of tests with regard to an execution coverage, e.g., path coverage, statement coverage, Boolean condition coverage, etc. A non-covered portion identifier identifies the non-covered portion, e.g., by noting breakpoint misses or static analysis constraint solution computational requirements. A trigger inserter inserts a trigger in the program, at a position which permits capture of data which aids exercise of the non-covered portion. A data capturer captures such execution coverage expansion data ("ECE data") in response to trigger activation. Captured ECE data is automatically made available in the test environment, thus expanding code execution coverage of the program in the test environment.