DFA test case generation system and method based on vertical large language model
Through the DFA test case generation system based on the vertical large language model, the limitations of the general large language model in the vertical field are solved, and the test case generation with high accuracy and coverage is achieved, which is suitable for actual enterprise testing needs.
Patent Information
- Application Number
- CN202411672247.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-21
- Publication Date
- 2025-03-11
AI Technical Summary
The general large language model shows limitations in specific vertical applications, including low interpretability, low adjustability, high bit error rate, low accuracy and low coverage rate, which cannot meet the needs of actual enterprise applications.
A DFA test case generation system based on a vertical large language model is adopted. The system includes input module, preprocessing module, DFA construction module and test case generation module. It is preprocessed and DFA construction through Transformer and BERT word segmenter to generate test cases that comply with professional specifications.
Reduce the bit error rate, improve the accuracy and coverage of test cases, ensure that the generated test cases can be directly applied to the actual test process, and improve the interpretability and adjustability of the results.
Smart Images
Figure CN119597655A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a DFA test case generation system and method based on a vertical large language model. Background Art
[0002] When working with enterprise customers, we found that general large language model technology showed obvious limitations in specific field applications. First, because general large language models are essentially black box models, their output results are difficult to interpret and users find it difficult to understand how the model draws conclusions. In addition, these models have low adjustability and are difficult to adjust according to specific needs, resulting in high bit error rates and affecting the reliability of the results.
[0003] Secondly, the general large language model has low accuracy and coverage when dealing with vertical domain tasks. Due to the lack of expertise in specific fields, the results generated by the model are not accurate or comprehensive enough to meet the actual enterprise application needs. This makes it difficult to use the generated test cases directly in software and hardware testing, limiting its actual value.
[0004] In addition, general large language models lack the understanding of vertical domain expertise. Insufficient grasp of industry norms, technical details, and professional terminology in vertical fields will result in the generated content not meeting industry standards, reducing the practicality of the model.
[0005] Therefore, in view of the above shortcomings, a DFA (deterministic finite state machine) test case generation system and method based on a vertical large language model are provided. Summary of the invention
[0006] The purpose of the present invention is to overcome the existing defects and provide a DFA test case generation system and method based on a vertical large language model, which not only reduces the bit error rate, but also improves the accuracy and coverage of the results, ensuring that the generated test cases can be directly applied to the actual test process.
[0007] The technical solution to achieve the above purpose is:
[0008] One of the present inventions is a DFA test case generation system based on a vertical large language model, comprising:
[0009] Input module, used to input test specification documents and read data in the documents;
[0010] The preprocessing module is used to represent the BERT (a pre-trained language generation model) tokenizer through the bidirectional encoder of the Transformer (transformer), which is a small model fine-tuned and trained specifically for processing and marking Chinese test specification documents, to complete the task of preprocessing documents;
[0011] The DFA construction module is used to convert the unstructured data in the preprocessed test specification document into a dedicated coarse-grained DFA process framework with variables, states and state transition information to generate test cases, and to generate a stable and usable DFA by fine-tuning the established DFA process generation model, and then output the DFA process data of the function points of the test specification document;
[0012] The test case generation module is used to convert the function point DFA process data of the output test specification document into test cases that comply with professional test case design specifications, functional logic, and professional language through the established test case generation model.
[0013] Preferably, in the preprocessing module, the content of the preprocessed document includes but is not limited to processing data loading, word segmentation, special tag processing and segment encoding within the document; converting the data text into input ID, attention mask and type ID and adding custom special tags through the BERT word segmenter.
[0014] Preferably, in the DFA construction module, variables are extracted from the "enabling conditions" section of the test specification document and the causal relationship in the text; wherein, for numerical variables, the DFA process generates a large model that divides the value range according to the threshold value in the document;
[0015] Each DFA state is assigned a unique ID, state type, and necessary comments. There are three types of states: initial state, general state, and terminal state.
[0016] State transition information includes the content and conditions of source state, target state, condition input, action input, change input and defined output.
[0017] Preferably, it also includes:
[0018] DFA optimization module is used to supplement boundary conditions, logic, variables, inputs and outputs, optimize the coarse-grained DFA framework, turn it into a fine-grained DFA framework, and then re-output the function point DFA process data of the test specification document;
[0019] The test case design strategy library is used to help the test case generate a large model to determine the specific use case design specifications.
[0020] A second method of the present invention for generating a DFA test case based on a vertical large language model comprises:
[0021] Step S1, input the test specification document and read the data in the document;
[0022] Step S2, using the Transformer's bidirectional encoder to represent the BERT word segmenter, a small model that is fine-tuned and trained specifically for processing and marking Chinese test specification documents, to complete the document preprocessing task;
[0023] Step S3, converting the unstructured data in the preprocessed test specification document into a dedicated coarse-grained DFA process framework with variables, states and state transition information to generate test cases, and generating a stable and usable DFA by fine-tuning the established DFA process generation large model, and then outputting the function point DFA process data of the test specification document;
[0024] Step S4, using the established test case generation model to transform the function point DFA process data of the output test specification document into test cases that comply with professional test case design specifications, functional logic, and professional language.
[0025] Preferably, in step S3, the variables are extracted from the "enabling conditions" part of the test specification document and the causal relationship in the text; wherein, for numerical variables, the DFA process generates a large model that divides the value range according to the threshold value in the document;
[0026] Each DFA state is assigned a unique ID, state type, and necessary comments. There are three types of states: initial state, general state, and terminal state.
[0027] State transition information includes the content and conditions of source state, target state, condition input, action input, change input and defined output.
[0028] Preferably, in step S3, the constructed coarse-grained DFA framework process can be optimized, and the optimization content includes: additionally supplementing boundary conditions, logic, variables, inputs and outputs, and then optimizing the coarse-grained DFA process framework into a fine-grained DFA process framework, and then re-outputting the functional point DFA process data of the test specification document.
[0029] Preferably, in step S4, the test case generation large model determines specific test case design specifications by referring to the test case design strategy library.
[0030] The beneficial effects of the present invention are as follows: the present invention completes the document preprocessing task through a small model, converts the unstructured data in the test specification document into a dedicated function-determined finite state DFA process framework with variables, states and state transitions for generating test cases, and generates a stable and usable deterministic finite state machine by fine-tuning the DFA process generation large model, thereby enhancing its knowledge and capabilities in vertical testing and improving the interpretability and adjustability of the output results; and converts the function point DFA process data of the output test specification document into a test case that conforms to the professional test case design specification, functional logic, and professional language through the established test case generation large model; this not only reduces the bit error rate, but also improves the accuracy and coverage of the results, ensuring that the generated test cases can be directly applied to the actual testing process. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] Figure 1 It is a module diagram of a DFA test case generation system based on a vertical large language model of the present invention;
[0032] Figure 2 The present invention is a flowchart of a DFA test case generation method based on a vertical large language model. DETAILED DESCRIPTION
[0033] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. In the description of the present invention, it should be noted that the terms "center", "up", "down", "left", "right", "vertical", "horizontal", "inside", "outside" and the like indicate directions or positional relationships based on the directions or positional relationships shown in the accompanying drawings, which are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore cannot be understood as a limitation on the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance.
[0034] The present invention will be further described below in conjunction with the accompanying drawings.
[0035] like Figure 1 As shown, a DFA test case generation system based on a vertical large language model includes: an input module 1, a preprocessing module 2, a DFA construction module 3, a test case generation module 4, a DFA optimization module 5 and a test case design strategy library 6.
[0036] Input module 1 is used to input the test specification document and read the data in the document.
[0037] Preprocessing module 2 is used to complete the task of preprocessing documents by using the Transformer's bidirectional encoder to represent the BERT word segmenter, a small model that is fine-tuned and trained specifically for processing and marking Chinese test specification documents.
[0038] In the embodiment, preprocessing the content of the document includes but is not limited to processing data loading, word segmentation, special tag processing and segment encoding within the document; converting the data text into input ID, attention mask and type ID and adding custom special tags through the BERT word segmenter, for example, [INDEX], [TITLE], [TITLE], [Content], [Previous], [Label] represent the document structure, data loading reads data from the JSONL file, builds a small model data set, initializes the small model, loads the pre-trained small model, and adjusts the word embedding size. Compared with the large language model, this small model has better task execution speed, effect, and stability.
[0039] DFA construction module 3 is used to convert the unstructured data in the preprocessed test specification document into an exclusive coarse-grained DFA process framework with variables, states and state transition information to generate test cases, and generate a stable and usable DFA from the coarse-grained DFA process framework by fine-tuning the established DFA process generation model, and then output the functional point DFA process data of the test specification document.
[0040] In the embodiment, variables are extracted from the "enabling conditions" section of the test specification document and the causal relationship in the text; wherein, for numerical variables, the DFA process generates a large model that divides the value range according to the threshold value in the document;
[0041] Each DFA state is assigned a unique ID, state type, and necessary comments. There are three types of states: initial state, general state, and terminal state.
[0042] State transition information includes the content and conditions of source state, target state, condition input, action input, change input and defined output.
[0043] In the embodiment, the DFA process generates a large model which is fine-tuned and trained with exclusive DFA annotated data, with the goal of improving the large model's ability to recognize and generate DFA format data.
[0044] The test case generation module 4 is used to transform the function point DFA process data of the output test specification document into test cases that meet professional test case design specifications, functional logic, and professional language through the established test case generation large model.
[0045] DFA optimization module 5 is used to supplement boundary conditions, logic, variables, inputs and outputs, optimize the coarse-grained DFA framework, and convert it into a fine-grained DFA framework. The fine-grained DFA process framework is used to generate a stable and usable DFA by fine-tuning the established DFA process generation model, and then re-output the functional point DFA process data of the test specification document.
[0046] The test case design strategy library 6 is used to help the test case generate a large model to determine the specific use case design specifications.
[0047] like Figure 2 As shown, a DFA test case generation method based on a vertical large language model includes:
[0048] Step S1, input the test specification document and read the data in the document.
[0049] Step S2, through the Transformer's bidirectional encoder, represents the BERT word segmenter, a small model that is fine-tuned and trained specifically for processing and marking Chinese test specification documents, to complete the document preprocessing task.
[0050] Step S3, convert the unstructured data in the preprocessed test specification document into an exclusive coarse-grained DFA process framework with variables, states and state transition information to generate test cases, and generate a stable and usable DFA from the coarse-grained DFA process framework by fine-tuning the established DFA process generation model, and then output the functional point DFA process data of the test specification document.
[0051] In the embodiment, variables are extracted from the "enabling conditions" section of the test specification document and the causal relationship in the text; wherein, for numerical variables, the DFA process generates a large model that divides the value range according to the threshold value in the document;
[0052] Each DFA state is assigned a unique ID, state type, and necessary comments. There are three types of states: initial state, general state, and terminal state.
[0053] State transition information includes the content and conditions of source state, target state, condition input, action input, change input and defined output.
[0054] In an embodiment, the constructed coarse-grained DFA framework process can be optimized, and the optimization content includes: additionally supplementing boundary conditions, logic, variables, inputs and outputs, and then optimizing the coarse-grained DFA process framework into a fine-grained DFA process framework, and generating a stable and usable DFA from the fine-grained DFA process framework by fine-tuning the established DFA process generation large model, and then re-outputting the functional point DFA process data of the test specification document.
[0055] Step S4, using the established test case generation model to transform the function point DFA process data of the output test specification document into test cases that comply with professional test case design specifications, functional logic, and professional language.
[0056] In the embodiment, the test case generation model determines the specific test case design specification by referring to the test case design strategy library.
[0057] Compared with the general large model generation test case based on the Transformer architecture, the deterministic finite state machine DFA generation test case based on the vertical large language model shows significant advantages including:
[0058] First, we can deeply understand the professional terms and specifications in vertical professional fields, which means that we can generate more accurate and effective test cases for the specific needs of specific industries. This in-depth understanding of professional fields improves the accuracy and coverage of tests when generating test cases, which can be directly used in business.
[0059] Secondly, the generation process is transparent and adjustable. This means that users can not only clearly understand the logic and process of test case generation, but also adjust the generation parameters as needed to adapt to different test requirements and environments. This flexibility and controllability is difficult to provide with general large models, whose internal working mechanisms are not transparent and difficult for enterprise customers to adjust and optimize.
[0060] In summary, the deterministic finite state machine DFA generation test case based on the vertical large language model can provide more accurate, comprehensive and practical test cases compared to the traditional black box large model through its deep understanding of the vertical field and the transparent adjustability of the generation process, thereby effectively improving the quality and efficiency of test writing.
[0061] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions described in the above embodiments may still be modified, or some or all of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. A DFA test case generation system based on a vertical large language model, characterized in that: include: Input module, used to input test specification documents and read data in the documents; The preprocessing module is used to represent the small model of the BERT tokenizer that is fine-tuned and trained specifically for processing and marking Chinese test specification documents through the Transformer's bidirectional encoder, thus completing the task of preprocessing test specification documents; The DFA construction module is used to convert the unstructured data in the preprocessed test specification document into a dedicated coarse-grained DFA process framework with variables, states and state transition information to generate test cases, and to generate a stable and usable DFA by fine-tuning the established DFA process generation model, and then output the DFA process data of the function points of the test specification document; The test case generation module is used to convert the function point DFA process data of the output test specification document into test cases that comply with professional test case design specifications, functional logic, and professional language through the established test case generation model.
2. A DFA test case generation system based on a vertical large language model according to claim 1, characterized in that: In the preprocessing module, the content of the preprocessed document includes but is not limited to processing data loading, word segmentation, special tag processing and segment encoding within the document; converting the data text into input ID, attention mask and type ID and adding custom special tags through the BERT word segmenter.
3. A DFA test case generation system based on a vertical large language model according to claim 1, characterized in that: In the DFA construction module, variables are extracted from the "enabling conditions" section of the test specification document and the causal relationship in the text; for numerical variables, the DFA process generates a large model that divides the value range according to the threshold in the document; Each DFA state is assigned a unique ID, state type, and necessary comments. There are three types of states: initial state, general state, and terminal state. State transition information includes the content and conditions of source state, target state, condition input, action input, change input and defined output.
4. A DFA test case generation system based on a vertical large language model according to claim 3, characterized in that: Also includes: DFA optimization module is used to supplement boundary conditions, logic, variables, inputs and outputs, optimize the coarse-grained DFA process framework, turn it into a fine-grained DFA process framework, and then re-output the function point DFA process data of the test specification document; The test case design strategy library is used to help the test case generate a large model to determine the specific use case design specifications.
5. A DFA test case generation method based on a vertical large language model, characterized in that: include: Step S1, input the test specification document and read the data in the document; Step S2, using the Transformer's bidirectional encoder to represent the BERT word segmenter, a small model that is fine-tuned and trained specifically for processing and marking Chinese test specification documents, to complete the document preprocessing task; Step S3, converting the unstructured data in the preprocessed test specification document into a dedicated coarse-grained DFA process framework with variables, states and state transition information to generate test cases, and generating a stable and usable DFA by fine-tuning the established DFA process generation large model, and then outputting the function point DFA process data of the test specification document; Step S4, using the established test case generation model to transform the function point DFA process data of the output test specification document into test cases that comply with professional test case design specifications, functional logic, and professional language.
6. A DFA test case generation method based on a vertical large language model according to claim 5, characterized in that: In step S3, variables are extracted from the "enabling conditions" part of the test specification document and the causal relationship in the text; wherein, for numerical variables, the DFA process generates a large model that divides the value range according to the threshold value in the document; Each DFA state is assigned a unique ID, state type, and necessary comments. There are three types of states: initial state, general state, and terminal state. State transition information includes the content and conditions of source state, target state, condition input, action input, change input and defined output.
7. The DFA test case generation method based on a vertical large language model according to claim 5, characterized in that: In step S3, the constructed coarse-grained DFA process framework can be optimized, and the optimization content includes: additionally supplementing boundary conditions, logic, variables, inputs and outputs, and then optimizing the coarse-grained DFA process framework into a fine-grained DFA process framework, and then re-outputting the functional point DFA process data of the test specification document.
8. The DFA test case generation method based on a vertical large language model according to claim 5, characterized in that: In step S4, the test case generation model determines the specific test case design specification by referring to the test case design strategy library.