Test set generation method and device of search enhancement generation system, computer device, readable storage medium and program product
By decoupling the retrieval enhancement system, the test difficulty is divided into several sub-dimensions. A scenario and difficulty matrix is constructed and an interactive confirmation mechanism is introduced. This solves the problems of difficulty in controlling the test case difficulty gradient and lack of business scenario adaptation in existing technologies, and achieves high-quality test set generation.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- GUANGZHOU QUWAN NETWORK TECH CO LTD
- Filing Date
- 2026-03-17
- Publication Date
- 2026-06-09
AI Technical Summary
Existing methods for generating test sets in search enhancement systems struggle to control the difficulty gradient of test cases, lack precise adaptation to business scenarios, and lack human-computer collaborative interaction and continuous optimization mechanisms, resulting in low test set quality.
By decoupling the retrieval enhancement system, the test difficulty is divided into several difficulty sub-dimensions. A scenario and difficulty matrix is constructed, a test set generation plan is generated, and an interactive confirmation mechanism is introduced to ensure that test cases follow the specified difficulty sub-dimension level and business scenario. Test sets are generated in combination with a large language model and knowledge base.
It achieves precise control of the difficulty gradient of test cases and improves scenario adaptability, thereby improving the quality of the test set, reducing labor costs and waste of computing resources, and enhancing the reliability and applicability of the test set.
Smart Images
Figure CN122173401A_ABST
Abstract
Claims
1. A method for generating a test set for a retrieval enhancement generation system, characterized in that, The method includes: Obtain several decoupled sub-dimensions of the test difficulty of the search enhancement generation system; Based on the level ranges corresponding to the aforementioned multiple difficulty sub-dimensions and the multiple business scenarios of the retrieval enhancement generation system, a scenario and difficulty matrix is obtained; Based on the scenario and difficulty matrix, test set generation requirements, and large language model, a test set generation plan is obtained; According to the test set generation plan, several test cases are generated to obtain the test set of the retrieval enhancement generation system.
2. The method according to claim 1, characterized in that, Before obtaining the test set generation plan based on the scenario and difficulty matrix, test set generation requirements, and large language model, the method further includes: Get the list of business scenarios, the allowable level range for each of the several difficulty sub-dimensions, the functional constraint information of test points, the test preference description information, and the total number of test cases generated; Based on the business scenario list, the allowable level range corresponding to each of the several difficulty sub-dimensions, the functional constraint information of the test points, the test preference description information, and the total number of test cases generated, the test set generation requirements are obtained.
3. The method according to claim 1, characterized in that, The step of generating several test cases according to the test set generation plan includes: Show the test set generation plan; In response to the tester's adjustment operation on the test set generation plan, an adjusted test set generation plan is obtained; Based on the adjusted test set generation plan, several test cases are generated.
4. The method according to claim 3, characterized in that, The step involves generating several test cases based on the adjusted test set generation plan, including: Based on the adjusted test set generation plan, the search keywords are obtained; Based on the search keywords, retrieve text fragments from the knowledge base that meet the similarity criteria; Based on the description information of the intelligent agent under test, the text fragments that meet the similarity conditions, the business scenario information in the adjusted test set generation plan, the level corresponding to the difficulty sub-dimension, and the functional constraint information of the test point, several initial test cases are generated. The initial test cases are deduplicated and padded to obtain a number of test cases.
5. The method according to claim 1, characterized in that, Based on several test cases, a test set is obtained, including: Obtain the review results from the testers regarding the aforementioned test cases; From the review results, the review decision distribution information, the non-compliance reason distribution information, and the non-compliance examples are extracted to obtain the review feedback information; Based on the review feedback information, rule patches are obtained; the rule patches include global rule patches and scenario rule patches; Based on the description information of the tested intelligent agent, the text fragments that meet the similarity conditions, the business scenario information in the rule patch and the adjusted test set generation plan, the level corresponding to the difficulty sub-dimension, and the functional constraint information of the test point, test cases are regenerated to obtain several optimized test cases. Based on the optimized test cases, a test set is obtained.
6. The method according to claim 1, characterized in that, The difficulty sub-dimension includes at least one of the retrieval sub-dimension and the reasoning sub-dimension.
7. A test set generation apparatus for a retrieval enhancement generation system, characterized in that, The device includes: The difficulty sub-dimension acquisition module is used to acquire several difficulty sub-dimensions of the test difficulty of the retrieval enhancement generation system after decoupling. The scenario and difficulty matrix acquisition module is used to obtain the scenario and difficulty matrix based on the level range corresponding to the several difficulty sub-dimensions and several business scenarios of the retrieval enhancement generation system. The test set generation plan acquisition module is used to obtain the test set generation plan based on the scenario and difficulty matrix, test set generation requirements and large language model; The test set acquisition module is used to generate several test cases according to the test set generation plan to obtain the test set of the retrieval enhancement generation system.
8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.
10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.