Interactive Software Repair via Human-Generated Test Oracles
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Automated software repair systems often produce incorrect repairs due to incomplete test suites, as they primarily check if the software program passes or fails test cases without ensuring the correctness of the repairs.
Innovation Solution
The integration of human-generated test oracles with automatically generated tests to create an augmented test suite, which interactsively identifies and corrects faults by generating and testing potential repairs, ensuring the software program meets the expected behavior.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If automated repair systems are used to identify and correct faults in software programs, then the productivity of fault correction is improved, but the reliability of the repairs deteriorates due to incomplete test suites
Solution Approach 1:
The patent combines automated repair systems with human-generated test oracles to create a hybrid approach. The automated system generates potential repairs efficiently, while human experts provide high-quality test oracles to verify correctness, thus merging the speed of automation with the reliability of human judgment.
Solution Approach 2:
Human-generated test oracles serve as an intermediary between the automated repair system and the final repaired software. These oracles act as a verification layer that mediates the acceptance of repairs, ensuring that automated repairs meet human-defined correctness criteria before being deployed.
2Reliability
If human-generated test oracles are added to automatically generated tests, then the reliability of repair verification is improved, but the device complexity increases
Solution Approach 1:
The test suite is segmented into two distinct components: automatically generated tests that check basic functionality and human-generated test oracles that verify correctness against expert knowledge. This segmentation allows each component to serve its specific purpose without overwhelming complexity.
Solution Approach 2:
Rather than requiring complete human review of all tests, the system applies human-generated oracles selectively to the most critical verification points. This partial action approach provides sufficient reliability improvement without the excessive complexity of full manual testing.
3Measurement precision
If the test suite is augmented with human-provided test oracles, then the measurement precision of repair correctness is improved, but the loss of time in test execution increases
Solution Approach 1:
Human-generated test oracles are prepared in advance and integrated into the test suite before repair verification. This preliminary action allows the system to quickly execute pre-defined correctness checks without requiring time-consuming human review during the actual repair validation process.
Solution Approach 2:
The augmented test suite performs self-verification using the integrated human-generated oracles. Once the oracles are established, the system automatically executes them against potential repairs without requiring ongoing human intervention, thus maintaining high measurement precision while minimizing time loss.
Data Source
AI summary
In some examples, a method to interactively repair a software program using one or more automatically generated tests with human-provided test oracles may include identifying a fault location in a software program, generating a potential repair at the fault location based on a repair candidate, automatically generating a first test to test the potential repair, and generating a first query for a first test oracle based on the first test. The method may also include obtaining a response to the first query from a human, generating a first human-provided test oracle based on the first query and the obtained response to the first query, augmenting a test suite to include the first automatically generated test with the first human-provided test oracle, and testing the potential repair using the augmented test suite including the first automatically generated test with the first human-provided test oracle.