面向智能问答系统逻辑推理功能的蜕变测试方法及系统
By constructing metamorphic relationships between word groups with multiple hops and multiple word groups with single hops, and using large-scale language models (LLM) for logical replacement, the problem of insufficient detection of logical reasoning ability in existing question-answering system testing is solved. This achieves efficient, diverse, and semantically consistent test case generation, thereby improving the testing effect of question-answering systems.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- TIANJIN UNIV
- Filing Date
- 2023-05-31
- Publication Date
- 2026-07-17
AI Technical Summary
Existing question-answering system testing methods cannot effectively detect logical reasoning ability, and existing metamorphosis testing frameworks have problems such as high false positives, narrow scope, and introduction of redundant semantic information, which cannot comprehensively test the logical reasoning function of question-answering systems.
By constructing metamorphic relationships between word groups with multiple hops and multiple word groups with single hops, test cases with high diversity and semantic invariance are generated. Logical substitution is performed using a large-scale language model (LLM) to ensure the naturalness and semantic consistency of test cases and improve the sufficiency of testing logical reasoning ability.
Test cases with strong error detection capabilities and efficient verification of model logic reasoning were generated, improving the sufficiency of the question-answering system's testing, reducing the false positive rate, and expanding the detection range.
Smart Images
Figure CN117009213B_ABST