LLM-Guided Unit Test Generation Through Coverage Feedback
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Generating unit tests for software functions is a tedious and labor-intensive task, often avoided by developers, and existing automated methods do not guarantee comprehensive code coverage without requiring developer input to create parameter matrices.
Innovation Solution
A computer-implemented method using a Large Language Model (LLM) to iteratively generate unit tests by selecting code elements, generating initial and corrected parameter values, and simulating function execution to ensure code coverage, without additional training of the LLM.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If automated unit test generation is implemented using traditional methods, then developer effort is reduced, but code coverage cannot be guaranteed and additional developer input is required
Solution Approach 1:
The system implements feedback by simulating function execution with generated test inputs and checking whether code elements are activated. Based on the simulation results, the system iteratively refines test input generation to achieve desired code coverage, creating a closed-loop automated testing process that guarantees coverage without developer intervention
Solution Approach 2:
The system performs self-service by automatically generating test inputs, simulating execution, analyzing coverage, and iteratively improving test cases without requiring developer input. The automated system serves itself to complete the entire unit test generation process, achieving both high automation and reliable code coverage
2Reliability
If manual unit test generation is performed by developers, then code coverage can be ensured, but the task becomes tedious and labor-intensive
Solution Approach 1:
The system replaces the mechanical process of manual test generation with an automated computational system that uses simulation and iterative refinement. This substitution eliminates the tedious manual work while maintaining code coverage guarantees, significantly improving developer productivity without sacrificing reliability
3Ease of manufacture
If parameter matrices are manually created for test generation, then test cases can be generated, but developer work is required and comprehensive coverage is not guaranteed
Solution Approach 1:
The system performs preliminary action by automatically analyzing the function source code to identify parameters and code elements before test generation. This preliminary analysis enables completely automated test case generation without requiring developers to manually create parameter matrices, while the subsequent simulation ensures comprehensive code coverage
Data Source
Figure 1
Figure 2~4b
Figure 5~6b
AI summary
Computer-implemented method for generating unit tests of a function, comprising obtaining initial values for the parameters of the function that activate a code element of the function, such that if the code element is not activated, the method iterates to obtain corrected values for those parameters that activate the code element, wherein obtaining the initial values and the corrected values of the parameters is done by interaction with an LLM.