Test case set generation method, device, equipment and computer readable storage medium
Through the combination of BERT model and knowledge graph, the automated generation of test case sets in the field of financial technology is achieved, the problem of inefficiency in the existing technology is solved, and the efficiency and accuracy of the generation of test case sets are improved.
Patent Information
- Application Number
- CN202010563141.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-06-18
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2040-06-18
AI Technical Summary
The existing technology is inefficient in the generation of test cases in the field of financial technology. It relies on manual writing and database retrieval, and cannot achieve true automated generation, resulting in low efficiency and inability to meet the high security and real-time requirements of the financial industry.
A test case set knowledge base constructed by BERT model combined with knowledge graphs is used to obtain case keywords for semantic analysis, similar case sets are retrieved, and a new test case set is generated using knowledge graph reasoning to achieve automated generation.
It improves the efficiency of the generation of test case sets, and can automatically infer new test case sets based on existing case libraries, improving the efficiency and accuracy of the development and testing process.
Smart Images

Figure CN111708703B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of financial technology (Fintech), and in particular to a test case set generation method, device, equipment and computer-readable storage medium. Background Art
[0002] With the development of computer technology, more and more technologies are being applied in the financial field. The traditional financial industry is gradually transforming into financial technology (Fintech). However, due to the security and real-time requirements of the financial industry, higher requirements are also placed on technology.
[0003] During the product development and testing process, product developers often need some test cases for development and testing. Test cases usually include usage scenarios and their corresponding test results. At present, test cases are mainly written and maintained manually, which is inefficient and requires a lot of manpower. At the same time, the existing automated case writing solution requires saving historical cases to the database first, and then generating a test case set through database retrieval and matching. Although there is a certain degree of automation, only existing cases can be retrieved during the database retrieval process, and the database cannot automatically generate new cases. The input of historical cases in the database needs to be manually entered and the retrieval scope of the cases is very limited, which is inefficient and cannot truly achieve the automated generation of test cases. Summary of the invention
[0004] The main purpose of the present invention is to provide a test case generation method, device, equipment and computer-readable storage medium, aiming to realize automatic generation of test cases and improve the generation efficiency of test cases.
[0005] To achieve the above object, the present invention provides a test case set generation method, the test case set generation method comprising:
[0006] Obtain case keywords, perform semantic analysis on the case keywords, and obtain semantic analysis results;
[0007] According to the semantic analysis result, a test case set knowledge base is searched to obtain a search result, wherein the test case set knowledge base is generated by training a preset training model obtained by combining a BERT model with a knowledge graph;
[0008] If it is determined according to the search result that a similar case set exists in the test case set knowledge base, then obtaining a target similar case set;
[0009] The knowledge graph is used to analyze the case keywords and the target similar case set to infer and generate a test case set.
[0010] Optionally, before the step of obtaining case keywords and performing semantic analysis on the case keywords to obtain semantic analysis results, the step further includes:
[0011] According to the unlabeled first training test case set, pre-processing training is performed on the preset training model to obtain an initial training model, wherein the preset training model is constructed based on the BERT model combined with the knowledge graph;
[0012] Fine-tune the initial training model according to the labeled second training test case set to obtain a language representation model;
[0013] The first training test case set and the second training test case set are classified by using the language representation model, and a test case set knowledge base is generated according to the classification results.
[0014] Optionally, the step of performing pre-processing training on a preset training model according to the unlabeled first training test case set to obtain an initial training model includes:
[0015] Obtaining first attribute information of a first unlabeled training test case set;
[0016] Dividing the first training test case set according to the first attribute information to obtain a plurality of first training test case subsets;
[0017] Pre-processing and training the preset training models according to the plurality of first training test case subsets respectively to obtain a plurality of corresponding initial training models, wherein the preset training models are constructed based on the BERT model combined with the knowledge graph;
[0018] The step of fine-tuning the initial training model according to the annotated second training test case set to obtain a trained language representation model comprises:
[0019] Obtaining a labeled second training test case set and second attribute information thereof;
[0020] Dividing according to the second attribute information and the second training test set to obtain a plurality of second training test case subsets;
[0021] Fine-tune the corresponding initial training models according to the plurality of second training test case subsets to obtain a plurality of language representation models corresponding to the first attribute information;
[0022] The step of classifying the first training test case set and the second training test case set according to the language representation model and generating a test case set knowledge base according to the classification results comprises:
[0023] The corresponding first training test case subset and the second training test case subset are classified by multiple language representation models to obtain multiple test case sets corresponding to the first attribute information, and a test case set knowledge base is generated based on the multiple test case sets.
[0024] Optionally, obtain candidate attribute information;
[0025] The step of searching the test case set knowledge base to obtain the search result according to the semantic analysis result comprises:
[0026] Determining a target test case set corresponding to the candidate attribute information in the test case set knowledge base;
[0027] The target test case set is retrieved according to the semantic analysis result to obtain the similarity between the case keyword and the test cases in the target test case set, and obtain a retrieval result.
[0028] Optionally, after the step of retrieving the target test case set according to the semantic analysis result to obtain the similarity between the case keyword and the test cases in the target test case set and obtaining the retrieval result, the step further includes:
[0029] Detecting whether the similarity in the search result is greater than or equal to a first preset threshold;
[0030] If the similarity is greater than or equal to the first preset threshold, it is determined that there is an identical case set corresponding to the case keyword in the test case set knowledge base, and the target identical case set is obtained and output.
[0031] Optionally, after the step of detecting whether the similarity in the search result is greater than or equal to a first preset threshold, the method further includes:
[0032] If the similarity is less than the first preset threshold, detecting whether the similarity is greater than a second preset threshold, wherein the second preset threshold is less than the first preset threshold;
[0033] If the similarity is greater than the second preset threshold, it is determined that there is a similar case set in the test case set knowledge base, and the steps are executed: obtaining a target similar case set;
[0034] If the similarity is less than or equal to the second preset threshold, a prompt message is output to prompt the user to manually generate a test case set.
[0035] Optionally, after the step of outputting prompt information to prompt the user to manually generate a test case set, the step further includes:
[0036] Obtain a set of annotated test cases manually generated by users;
[0037] The test case set knowledge base is updated according to the annotated test case set.
[0038] In addition, to achieve the above-mentioned purpose, the present invention also provides a test case set generation device, the test case set generation device comprising:
[0039] An analysis module is used to obtain case keywords, perform semantic analysis on the case keywords, and obtain semantic analysis results;
[0040] A retrieval module, used to retrieve a test case set knowledge base to obtain a retrieval result according to the semantic analysis result, wherein the test case set knowledge base is generated by training a preset training model obtained by combining a BERT model with a knowledge graph;
[0041] A first acquisition module, configured to acquire a target similar case set if it is determined that a similar case set exists in the test case set knowledge base according to the search result;
[0042] The first generation module is used to analyze the case keywords and the target similar case set by using the knowledge graph to generate a test case set.
[0043] In addition, to achieve the above-mentioned purpose, the present invention also provides a test case set generation device, which includes: a memory, a processor, and a test case set generation program stored on the memory and executable on the processor, and when the test case set generation program is executed by the processor, the steps of the test case set generation method as described above are implemented.
[0044] In addition, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, on which a test case set generation program is stored, and when the test case set generation program is executed by a processor, the steps of the test case set generation method as described above are implemented.
[0045] The present invention provides a test case set method, device, equipment and computer-readable storage medium, which obtains case keywords, performs semantic analysis on the case keywords, and obtains semantic analysis results; according to the semantic analysis results, retrieves the test case set knowledge base to obtain retrieval results, wherein the test case set knowledge base is generated by training a preset training model obtained by combining a BERT model with a knowledge graph; if it is determined according to the retrieval results that there is a similar case set in the test case set knowledge base, then obtain a target similar case set; use the knowledge graph to analyze the case keywords and the target similar case set to generate a test case set. In the above manner, the existing test case set knowledge base can be used to search in the test case set knowledge base according to the case keywords to obtain retrieval results, and then, according to the target similar case set in the retrieval results, a new test case set is automatically inferred and generated using the knowledge graph. Therefore, compared with the prior art, the present invention can automatically infer and generate a new test case set based on the existing test case set knowledge base, thereby improving the generation efficiency of the test case set. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] Figure 1 A schematic diagram of the device structure of the hardware operating environment involved in the embodiment of the present invention;
[0047] Figure 2 A schematic diagram of a flow chart of a first embodiment of a method for generating a test case set according to the present invention;
[0048] Figure 3 It is a functional module diagram of the first embodiment of the test case set generating device of the present invention.
[0049] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION
[0050] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.
[0051] Reference Figure 1 , Figure 1 The figure is a schematic diagram of the device structure of the hardware operating environment involved in the embodiment of the present invention.
[0052] The test case set generating device in the embodiment of the present invention may be a smart phone, or may be a terminal device such as a PC (Personal Computer), a tablet computer, or a portable computer.
[0053] like Figure 1As shown, the test case set generation device may include: a processor 1001, such as a CPU, a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a Wi-Fi interface). The memory 1005 may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.
[0054] Those skilled in the art will understand that Figure 1 The test case set generating device structure shown in the figure does not constitute a limitation on the test case set generating device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.
[0055] like Figure 1 As shown, the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a test case set generation program.
[0056] exist Figure 1 In the terminal shown, the network interface 1004 is mainly used to connect to the background server and communicate data with the background server; the user interface 1003 is mainly used to connect to the client and communicate data with the client; and the processor 1001 can be used to call the test case set generation program stored in the memory 1005, and execute the various steps of the following test case set generation method.
[0057] Based on the above hardware structure, various embodiments of the test case set generation method of the present invention are proposed.
[0058] The invention provides a test case set generation method.
[0059] Reference Figure 2 , Figure 2 It is a flowchart of the first embodiment of the test case set generation method of the present invention.
[0060] In this embodiment, the test case set generation method includes:
[0061] Step S10, obtaining case keywords, performing semantic analysis on the case keywords, and obtaining semantic analysis results;
[0062] The test case set generation method of this embodiment is implemented by a test case set generation device, and the device is equipped with a test case set generator.
[0063] In this embodiment, the test case set is some test scenarios and corresponding expected test results that developers and testers may use during the product development process. Optionally, it includes relevant test information of some abnormal scenarios. For example, for a login interface, the test case set can include different usage scenarios such as Chinese login name, English login name, special character login name, etc., and also includes the test results when testing in these different test scenarios. When it is necessary to generate a new test case set, the staff can input the case keywords of the test case set to be generated through the corresponding software on the working end, such as "login", to trigger the test case set generation instruction. At this time, when the test case set generator receives the test case set generation instruction, it obtains the case keywords input by the staff, and then performs semantic analysis on the case keywords to use semantic analysis to review the context-related properties, thereby obtaining the corresponding semantic analysis results, wherein the semantic analysis method used in this application is a commonly used semantic analysis method, such as latent semantic analysis.
[0064] Step S20, searching a test case set knowledge base to obtain a search result according to the semantic analysis result, wherein the test case set knowledge base is generated by training a preset training model obtained by combining a BERT model with a knowledge graph;
[0065] Then, according to the semantic analysis results, the test case set knowledge base is retrieved to obtain the retrieval results.
[0066] Among them, the test case set knowledge base contains all case sets related to the product, and the test case set knowledge base is generated by training the preset training model obtained by combining the BERT model with the knowledge graph. Among them, BERT (BidirectionalEncoder Representations from Transformers, a natural language processing pre-training technology based on neural networks) can handle natural language semantic analysis, classification and other scenes very well, but there are also some shortcomings, such as the lack of common sense. Test cases require a lot of test background knowledge support. What BERT learns is a text matching model. A lot of test background common sense is implicit and vague, which is difficult to reflect in pre-training data; at the same time, there is a lack of understanding of semantics and lack of reasoning ability. Therefore, the knowledge graph information is integrated into the pre-training process, and the knowledge in the test can be organized. The computational model based on symbolic semantics can provide BERT with prior knowledge, so that it has certain test common sense and reasoning ability. Therefore, in this application, BERT is used to train and generate a test case set knowledge base in combination with the preset training model of the knowledge graph, so that the test case set knowledge base can know different test common sense and generate a new test case set using test common sense reasoning.
[0067] The search results are the similarity between the case keywords and the existing cases in the test case set knowledge base after semantic analysis. According to the similarity, the search results generally include three different search results: exactly the same, similar, and basically different.
[0068] Step S30: If it is determined according to the search result that a similar case set exists in the test case set knowledge base, a target similar case set is obtained.
[0069] If it is determined according to the search results that there is a similar case set in the test case set knowledge base, then the target similar case set is obtained. Specifically, for the input case keyword, according to its semantic analysis result, it is compared with the test set in the test case set knowledge base to obtain the similarity with each test set, wherein the case with the greatest similarity among the similarity values with different test sets is taken as the final search result and the case set corresponding to the maximum similarity value is taken as the target similar case set.
[0070] Step S40, analyzing the case keywords and the target similar case set using the knowledge graph to generate a test case set;
[0071] Finally, the knowledge graph is used to analyze the case keywords and the target similar case set, and the test case set is generated by reasoning. The knowledge graph is a knowledge domain visualization or knowledge domain mapping map, which is a series of various graphics that show the knowledge development process and structural relationship. The reasoning of the knowledge graph includes deductive reasoning and inductive reasoning. Since inductive reasoning can add new knowledge, inductive reasoning is mainly used in this application. Inductive reasoning can also use the FOIL (First Order Inductive Learner) algorithm, the association rule mining algorithm of the incomplete knowledge base, and the path sorting algorithm. Specifically, one or more of the above algorithms are first used to learn or construct rules for the target similar case set, and then new entities are inferred according to the rules obtained by learning or constructing the case keywords and the entities in the target similar case set, and then the test case set can be constructed based on the new entities obtained by reasoning. If the case keyword entered is a login password, and the existing target similar case set is a login name, the login password and the login name can be used to infer and generate a test case set related to the login password.
[0072] The embodiment of the present invention provides a test case set generation method, which obtains case keywords, performs semantic analysis on the case keywords, and obtains semantic analysis results; according to the semantic analysis results, retrieves the test case set knowledge base to obtain retrieval results, wherein the test case set knowledge base is generated by training a preset training model obtained by combining a BERT model with a knowledge graph; if it is determined according to the retrieval results that there is a similar case set in the test case set knowledge base, then obtain a target similar case set; use the knowledge graph to analyze the case keywords and the target similar case set, and infer to generate a test case set. Through the above method, the existing test case set knowledge base can be used to search in the test case set knowledge base according to the case keywords to obtain retrieval results, and then, according to the target similar case set in the retrieval results, a new test case set is automatically inferred and generated using the knowledge graph. Therefore, compared with the prior art, the present invention can automatically infer and generate a new test case set based on the existing test case set knowledge base, thereby improving the generation efficiency of the test case set.
[0073] Further, based on Figure 2 The first embodiment shown provides a second embodiment of the test case set generation method of the present invention.
[0074] In this embodiment, before the above step S10, the test case set generation method further includes:
[0075] Step A, pre-processing and training a preset training model according to an unlabeled first training test case set to obtain an initial training model, wherein the preset training model is constructed based on a BERT model combined with a knowledge graph;
[0076] In this embodiment, the preset training model is first pre-processed and trained based on the unlabeled first training test case set to obtain an initial training model, wherein the preset training model is constructed based on the BERT model combined with the knowledge graph.
[0077] Among them, the first unlabeled training test case set is a previously stored test case set. The preset training model is constructed based on the BERT model combined with the knowledge graph. BERT can handle natural language semantic analysis, classification and other scenarios very well, but it also has some shortcomings, such as the lack of common sense. Test cases require a lot of test background knowledge support. BERT learns a text matching model. A lot of test background common sense is implicit and vague, which is difficult to reflect in pre-training data; at the same time, it lacks understanding of semantics and reasoning ability. Therefore, incorporating knowledge graph information into the pre-training process can organize the knowledge in the test. The computational model based on symbolic semantics can provide BERT with prior knowledge, so that it has certain test common sense and reasoning ability. BERT pre-trains on a large number of test case corpora to achieve understanding of the text semantics of test cases. Specifically, BERT first randomly hides some words in the test, and then realizes language representation through context prediction to obtain the initial training model. For example, for the sentence "Dylan wrote Blowing in the Wind in 1962 and Chronicles: Book 1 in 2004", BERT can randomly hide the words "Dylan", "1962", "Blowing in the Wind", etc., but through continuous training, the model can determine the relationship between these words and store the relationship between these words, so that the model can know the relationship between words, that is, language representation. It should be noted that before the preprocessing training process, BERT is combined with the knowledge graph, and the multi-information entities in the knowledge graph (such as Dylan and "The Answer is Blowing in the Wind" in the above example) are used as external knowledge to improve language representation, so that the model can know the meaning of each word itself, not just the relationship between multiple words. At the same time, structured knowledge encoding and heterogeneous information fusion are realized (wherein, the implementation method of structured knowledge encoding is: abstract knowledge is converted into vectors and other forms for language representation, but because the vector representation spaces of knowledge and text are different, the text and knowledge are fused in a unified feature space through a preset efficiency model; heterogeneous information refers to different types of information such as vocabulary, syntax and knowledge information). On the basis of the BERT model, the knowledge graph is integrated to build a preset training model. Specifically, abstract knowledge information needs to be encoded so that it can be used for language representation. At the same time, the encoding of words and knowledge during BERT pre-training are different. Although both are converted into vectors, they are located in different vector spaces. Therefore, it is necessary to design the model to achieve the fusion of heterogeneous information such as vocabulary, syntax and knowledge information. The BERT model combined with the knowledge graph can solve the above problems.
[0078] Step B, fine-tuning the initial training model according to the annotated second training test case set to obtain a language representation model;
[0079] Then, the initial training model is fine-tuned according to the labeled second training test case set to obtain a language representation model.
[0080] The annotated second training test set is a training test set that is not included in the original first training test case set, such as a test case set of a new login name. The annotated second training test case set can supplement the first training test set, thereby making the language representation model more comprehensive and building a test case set knowledge base that is more in line with the actual test scenario.
[0081] The fine-tuning training process is completed by two modules: the text encoder and the knowledge encoder. The text encoder is responsible for obtaining the semantic information such as lexical and syntactic information of the input tokens of the second training test case set, summing the token vector, segmentation vector and position vector to obtain the input vector, and then extracting semantic features through a multi-layer bidirectional conversion encoder. The knowledge encoder integrates additional entity-oriented knowledge information into the text information from the bottom layer, so that the heterogeneous information of tokens and entities can be represented in a unified feature space. 1 ,…,w n} represents the vector of the labeled sequence, and {e 1 ,…,e n} to represent the vector of entities in the sequence. The two sequences are calculated according to the following formula:
[0082]
[0083]
[0084] Among them, MH-ATT is the attention layer.
[0085] Then align the tokens in the sequence with the corresponding entities (the entity is aligned with the corresponding first token), and then input such a sequence into the information fusion layer. The calculation steps of the information fusion layer are as follows:
[0086]
[0087]
[0088]
[0089] Among them, h j represents the internal hidden state of the fusion tag and entity information, b represents the bias, and W t represents the weight in the hidden layer, and σ() is a nonlinear activation function.
[0090] Through the above-mentioned fine-tuning process, a language representation model is obtained, so that a test case set knowledge base can be obtained based on the language representation model in the subsequent process.
[0091] Step C, classifying the first training test case set and the second training test case set by using the language representation model, and generating a test case set knowledge base according to the classification results;
[0092] Finally, the first training test case set and the second training test case set are classified by the language representation model, and a test case set knowledge base is generated according to the classification results. Specifically, the language representation model obtains the classification of different training test case sets based on the probability distribution calculation formula, and then finally generates a test case set knowledge base, where the predicted probability distribution calculation formula is as follows:
[0093]
[0094] Among them, linear() represents the linear layer.
[0095] That is, each case in different training test case sets is classified, and cases belonging to the same category are combined into the same category of cases, and then a test case set knowledge base is constructed from cases of different categories. Then, for example, user names and passwords can all belong to login cases.
[0096] In this embodiment, according to the unlabeled first training test case set, the preset training model is preprocessed and trained to obtain an initial training model, wherein the preset training model is constructed based on the BERT model combined with the knowledge graph; the initial training model is fine-tuned and trained according to the labeled second training test case set to obtain a language representation model; the first training test case set and the second training test case set are classified by the language representation model, and a test case set knowledge base is generated according to the classification result. The training model is trained according to the pre-training process and the fine-tuning training process to obtain a test case set knowledge base, which realizes the classification of different test case sets, facilitates subsequent retrieval, and improves the efficiency of subsequent test case set generation. In addition, the model is trained through the pre-training process and the fine-tuning training process, and then the training test case set is classified according to the trained model, and finally a test case set knowledge base is generated. Compared with the traditional manually written test cases, the present invention is more appropriate for the case of the example, and the accuracy of test case generation is further improved.
[0097] Furthermore, based on the above embodiments, a third embodiment of the test case set generation method of the present invention is proposed.
[0098] In this embodiment, step A includes:
[0099] Step a1, obtaining first attribute information of a first unlabeled training test case set;
[0100] Step a2, dividing the first training test case set according to the first attribute information to obtain a plurality of first training test case subsets;
[0101] Step a3: pre-process and train the preset training model according to multiple first training test case subsets to obtain corresponding multiple initial training models, wherein the preset training model is constructed based on the BERT model combined with the knowledge graph.
[0102] In this embodiment, the training test case set (including the first training test case set and the second training test case set) can be divided according to the attribute information to train a language representation model corresponding to multiple different attribute information, and then classified in combination with the classification results of the language representation model and the attribute information, and a test case set knowledge base is constructed.
[0103] Specifically, firstly, first attribute information of an unlabeled first training test case set is obtained; then, the first training test case set is divided according to the first attribute information to obtain a plurality of first training test case subsets.
[0104] In this embodiment, the input source of the training model consists of four parts: a test knowledge common library, a bug database, a business scenario library, and a training database. The test knowledge common library mainly includes common test cases with business commonality, such as login, password verification, etc.; the bug database is a bug case found in production; the business scenario library is a collection of test cases written under specific business scenarios, and the training database is a set of test cases manually annotated on the TCTP platform. The training parameter case set will be trained according to the training data of the three dimensions of full product cases, specific project product cases, and personalized writing cases. Correspondingly, the first attribute information may include different attributes such as full product cases, specific project product cases, and personalized writing cases. Full product cases are test case sets for a category of products such as insurance, project product cases are test case sets for specific products such as login, and personalized insurance cases can be test case sets associated with each writer.
[0105] For the first training test case subsets with different attributes, the classification results of the same case in the initial training model formed therefrom may be different, and the association relationships between different entities may be different. Therefore, the preset training model is pre-processed and trained according to multiple first training test case subsets to obtain multiple corresponding initial training models, wherein the preset training model is constructed based on the BERT model combined with the knowledge graph.
[0106] At this time, step B includes:
[0107] Step b1, obtaining a labeled second training test case set and its second attribute information;
[0108] Step b2, dividing according to the second attribute information and the second training test set to obtain a plurality of second training test case subsets;
[0109] Step b3: fine-tune the corresponding initial training model according to multiple second training test case subsets to obtain multiple language representation models corresponding to the second attribute information.
[0110] For the annotated second training test case set, similar to the first training test case set, initial training models that match the second attribute information of the second training test case set are determined respectively, and the second attribute information in the second training test case set is added to the corresponding initial training model for fine-tuning training to obtain multiple language representation models.
[0111] Based on the above method, a language representation model is generated, and different language representation models are determined according to the attribute information, so that models with different attributes can be obtained. The classification results of the obtained language representation model are more accurate, and the diversity of the language representation model is increased to adapt to different test scenarios.
[0112] At this time, step C includes:
[0113] Step c1, classifying the corresponding first training test case subset and the second training test case subset by using multiple language representation models to obtain multiple test case sets corresponding to the first attribute information, and generating a test case set knowledge base based on the multiple test case sets;
[0114] The language representation models with multiple different attributes are combined to form the test case set knowledge base of the device. The test case set knowledge base includes cases, attributes (full product cases, specific project product cases, personalized product cases, personalized writing cases) and test sets. Different test sets will be classified into corresponding cases. At the same time, for the same test set, depending on the different attributes, different cases may correspond to different test case set sub-knowledge bases with different attributes.
[0115] In this embodiment, multiple speech representation models with different attributes are combined to form a final test case set knowledge base, thereby ensuring the completeness of the test case set knowledge base, and enabling the test case set to match more usage scenarios according to different attributes, further improving the accuracy of test case set generation. At the same time, when the staff conducts a search, the search scope can be narrowed based on the input alternative attribute information, thereby improving the retrieval efficiency and improving the generation efficiency of the test case set.
[0116] Furthermore, based on the above embodiments, a fourth embodiment of the test case set generation method of the present invention is proposed.
[0117] In this embodiment, after the above step S20, the test case set generation method further includes:
[0118] Step D, obtaining candidate attribute information;
[0119] In this embodiment, when triggering the test case set generation instruction, the staff can enter not only case keywords but also alternative attribute information, wherein the alternative attribute information is the attribute information corresponding to the test case set to be generated, and the alternative attribute information corresponds to the attribute information of each language representation model of the test case set knowledge base, that is, the alternative attribute information is used to give the associated test case set during subsequent retrieval. Correspondingly, the test case set generator can first obtain the alternative attribute information.
[0120] Step S20 includes:
[0121] Step E, determining a target test case set corresponding to the candidate attribute information in the test case set knowledge base;
[0122] Step F, searching the target test case set according to the semantic analysis result to obtain the similarity between the case keyword and the test case in the target test case set, and obtaining a search result;
[0123] Then, a target test case set corresponding to the candidate attribute information is determined in the test case set knowledge base, and then the target test case set is retrieved according to the semantic analysis result to obtain the similarity between the case keywords and the test cases in the target test case set, and obtain the retrieval result.
[0124] When one or more candidate attribute information is selected, the output result gives a test case set that meets the attribute based on the cases associated with the test attribute. At the same time, during the retrieval process, based on the selected candidate attribute information, only the language representation model in the test case set knowledge base with the same attribute information as the candidate attribute information will be retrieved. If the alternative attribute is a specific project product case set, only the test case set knowledge base with the attribute as the specific product case set will be retrieved, without retrieving all product cases and personalized writing cases. This can improve the efficiency of the retrieval process and make the retrieval results more accurate. The corresponding retrieval results are determined by the similarity between the case keywords and the test cases in the test case set of the corresponding attributes.
[0125] Through the above-mentioned manner, this embodiment can narrow down the scope of the test case set knowledge base to be searched according to the candidate attribute information during the search process, thereby improving the efficiency and accuracy of the search.
[0126] Furthermore, based on the above embodiments, a fifth embodiment of the test case set generation method of the present invention is proposed.
[0127] In this embodiment, step S20 includes:
[0128] Step G, detecting whether the similarity in the search result is greater than or equal to a first preset threshold;
[0129] Step H: if the similarity is greater than or equal to the first preset threshold, it is determined that there is an identical case set corresponding to the case keyword in the test case set knowledge base, and the target identical case set is obtained and output;
[0130] In this embodiment, it is detected whether the similarity in the retrieval results is greater than or equal to the first preset threshold. When the similarity is greater than or equal to the first preset threshold, it means that the current test case set knowledge base already has a case set with the same case keyword as the input case. In this way, the target identical case set is directly determined based on the similarity and output, and the required test case set can be output.
[0131] Furthermore, after step H, the method further includes:
[0132] Step I: if the similarity is less than the first preset threshold, detecting whether the similarity is greater than a second preset threshold, wherein the second preset threshold is less than the first preset threshold;
[0133] When the similarity is less than the first preset threshold, it means that there is no identical test case set in the test case set knowledge base, but the test case set knowledge base generated by the BERT combined with the knowledge graph training model itself has a certain learning ability. Therefore, the next step is to determine whether there are similar cases, that is, whether the similarity is greater than the second preset threshold.
[0134] Step J: if the similarity is greater than the second preset threshold, it is determined that there is a similar case set in the test case set knowledge base, and then step S30 is executed: obtaining a target similar case set;
[0135] When it is greater than the second preset threshold, although there is no identical test set in the test case set knowledge base, there is a similar case set associated with the case keyword. Then, based on the case keyword and the target similar case set, the reasoning ability of the knowledge graph is used to infer and generate the test case set. If the case keyword is a user password, and it is determined that there is a similar test set in the test case set knowledge base that is a test set related to the user name, such as one that does not contain special characters and is at least six characters long, then a test case set that does not contain special characters in the user password and is at least six characters long can be inferred and generated.
[0136] Step K: if the similarity is less than or equal to the second preset threshold, output a prompt message to prompt the user to manually generate a test case set;
[0137] When the similarity is less than or equal to the second preset threshold, it means that the test cases in the test case set knowledge base are quite different from the input case keywords, and the case corresponding to the input case keyword should be a brand new case. At this time, the existing test case set cannot be used for direct output or inference to generate a test case set. If the user needs to manually generate a test case set, the test case set is manually added.
[0138] Furthermore, after step K, the method further includes:
[0139] Step k1, obtaining a set of annotated test cases manually generated by a user;
[0140] Step k2, updating the test case set knowledge base according to the annotated test case set;
[0141] When the user manually inputs a new annotated test case set, the annotated test case set manually generated by the user is recorded, and the annotated test case set is input as a new annotated test case set to update the test case set knowledge base, so that the test case set knowledge base can learn according to the annotated test case set, thereby expanding the test case set knowledge base.
[0142] In this embodiment, the same test case set can be directly output according to the similarity based on the retrieval results, or a test case set can be generated by inference based on similar test case sets. When the test case set cannot be output according to the test case set knowledge base, an annotated test case set can be generated by artificial generation, and then the annotated test case set can be used to update the test case set knowledge base to expand the test case set knowledge base.
[0143] The invention also provides a test case set generating device.
[0144] Reference Figure 3 , Figure 3 It is a functional module diagram of the first embodiment of the test case set generating device of the present invention.
[0145] like Figure 3 As shown, the test case set generating device comprises:
[0146] An analysis module 10 is used to obtain case keywords, perform semantic analysis on the case keywords, and obtain semantic analysis results;
[0147] A retrieval module 20 is used to retrieve a test case set knowledge base to obtain a retrieval result according to the semantic analysis result, wherein the test case set knowledge base is generated by training a preset training model obtained by combining a BERT model with a knowledge graph;
[0148] A first acquisition module 30, configured to acquire a target similar case set if it is determined according to the search result that a similar case set exists in the test case set knowledge base;
[0149] The first generating module 40 is used to analyze the case keywords and the target similar case set by using the knowledge graph, and to generate a test case set by reasoning.
[0150] Furthermore, the test case set generating device further comprises:
[0151] A pre-training module, which performs pre-processing training on a preset training model according to an unlabeled first training test case set to obtain an initial training model, wherein the preset training model is constructed based on a BERT model combined with a knowledge graph;
[0152] A fine-tuning training module, used to perform fine-tuning training on the initial training model according to the annotated second training test case set to obtain a language representation model;
[0153] The second generating module is used to classify the first training test case set and the second training test case set by using the language representation model, and generate a test case set knowledge base according to the classification results.
[0154] Furthermore, the pre-training module also includes:
[0155] A first acquisition unit, used to acquire first attribute information of a first unlabeled training test case set;
[0156] A first division unit, configured to divide the first training test case set according to the first attribute information to obtain a plurality of first training test case subsets;
[0157] A pre-training unit, used to perform pre-processing training on a preset training model according to a plurality of first training test case subsets, respectively, to obtain a plurality of corresponding initial training models, wherein the preset training model is constructed based on a BERT model combined with a knowledge graph;
[0158] The fine-tuning training module also includes:
[0159] A second acquisition unit, used to acquire a labeled second training test case set and second attribute information thereof;
[0160] A second partitioning unit, configured to partition the second training and testing set according to the second attribute information and the second training and testing set to obtain a plurality of second training and testing case subsets;
[0161] A fine-tuning training unit, used to perform fine-tuning training on the corresponding initial training model according to the plurality of second training test case subsets, respectively, to obtain a plurality of language representation models corresponding to the second attribute information;
[0162] The second generation module also includes:
[0163] The first generating unit is used to classify the corresponding first training test case subset and the second training test case subset through multiple language representation models to obtain multiple test case sets corresponding to the first attribute information, and generate a test case set knowledge base based on the multiple test case sets.
[0164] Furthermore, the test case set generating device further comprises:
[0165] A second acquisition unit, used to acquire candidate attribute information;
[0166] The first acquisition module also includes:
[0167] A determination unit, configured to determine a target test case set corresponding to the candidate attribute information in the test case set knowledge base;
[0168] The third acquisition unit is used to retrieve the target test case set according to the semantic analysis result to obtain the similarity between the case keyword and the test case in the target test case set to obtain the search result.
[0169] Furthermore, the test case set generating device further comprises:
[0170] A first detection module, used to detect whether the similarity in the search result is greater than or equal to a first preset threshold;
[0171] The first output module is used to determine if the similarity is greater than or equal to the first preset threshold value, whether there is an identical case set corresponding to the case keyword in the test case set knowledge base, obtain the target identical case set, and output it.
[0172] Furthermore, the test case set generating device further comprises:
[0173] A second detection module, configured to detect whether the similarity is greater than a second preset threshold if the similarity is less than the first preset threshold, wherein the second preset threshold is less than the first preset threshold;
[0174] A fourth acquisition module is used for determining that there is a similar case set in the test case set knowledge base if the similarity is greater than the second preset threshold, and then executing the steps of: acquiring a target similar case set;
[0175] The second generating module is used to output a prompt message to prompt the user to manually generate a test case set if the similarity is less than or equal to the second preset threshold.
[0176] Furthermore, the test case set generating device further comprises:
[0177] A fifth acquisition module is used to acquire a set of annotated test cases manually generated by a user;
[0178] An updating module is used to update the test case set knowledge base according to the annotated test case set.
[0179] Among them, the functional implementation of each module in the above-mentioned test case set generation device corresponds to the various steps in the above-mentioned test case set generation method embodiment, and its functions and implementation processes are no longer repeated here one by one.
[0180] The present invention also provides a computer-readable storage medium on which a test case set generation program is stored. When the test case set generation program is executed by a processor, the steps of the test case set generation method described in any of the above embodiments are implemented.
[0181] The specific embodiments of the computer-readable storage medium of the present invention are substantially the same as the embodiments of the test case set generation method described above, and are not described in detail herein.
[0182] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.
[0183] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.
[0184] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present invention.
[0185] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.
Claims
1. A test case set generation method, characterized in that: The test case set generation method comprises: Obtain case keywords, perform semantic analysis on the case keywords, and obtain semantic analysis results; According to the semantic analysis result, a test case set knowledge base is searched to obtain a search result, wherein the test case set knowledge base is generated by training a preset training model obtained by combining a BERT model with a knowledge graph; If it is determined according to the search result that a similar case set exists in the test case set knowledge base, then obtaining a target similar case set; The knowledge graph is used to analyze the case keywords and the target similar case set to generate a test case set.
2. The test case set generation method according to claim 1, characterized in that: Before the step of obtaining case keywords and performing semantic analysis on the case keywords to obtain semantic analysis results, the method further includes: According to the unlabeled first training test case set, pre-processing training is performed on the preset training model to obtain an initial training model, wherein the preset training model is constructed based on the BERT model combined with the knowledge graph; Fine-tune the initial training model according to the labeled second training test case set to obtain a language representation model; The first training test case set and the second training test case set are classified by using the language representation model, and a test case set knowledge base is generated according to the classification results.
3. The test case set generation method according to claim 2, characterized in that: The step of performing pre-processing training on the preset training model according to the unlabeled first training test case set to obtain the initial training model comprises: Obtaining first attribute information of a first unlabeled training test case set; Dividing the first training test case set according to the first attribute information to obtain a plurality of first training test case subsets; Pre-processing and training the preset training models according to the plurality of first training test case subsets respectively to obtain a plurality of corresponding initial training models, wherein the preset training models are constructed based on the BERT model combined with the knowledge graph; The step of fine-tuning the initial training model according to the annotated second training test case set to obtain a trained language representation model comprises: Obtaining the labeled second training test case set and second attribute information thereof; Dividing according to the second attribute information and the second training test case set to obtain a plurality of second training test case subsets; Fine-tune the corresponding initial training models according to the plurality of second training test case subsets to obtain a plurality of language representation models corresponding to the second attribute information; The step of classifying the first training test case set and the second training test case set according to the language representation model and generating a test case set knowledge base according to the classification results comprises: The corresponding first training test case subset and the second training test case subset are classified by multiple language representation models to obtain multiple test case sets corresponding to the first attribute information, and a test case set knowledge base is generated based on the multiple test case sets.
4. The test case set generation method according to claim 3, characterized in that: Before the step of searching the test case set knowledge base to obtain the search result according to the semantic analysis result, the method further includes: Get the candidate attribute information; The step of searching the test case set knowledge base to obtain the search result according to the semantic analysis result comprises: Determining a target test case set corresponding to the candidate attribute information in the test case set knowledge base; The target test case set is retrieved according to the semantic analysis result to obtain the similarity between the case keyword and the test cases in the target test case set, and obtain a retrieval result.
5. The test case set generation method according to claim 4, characterized in that: After the step of searching the target test case set according to the semantic analysis result to obtain the similarity between the case keyword and the test case in the target test case set and obtaining the search result, the step further includes: Detecting whether the similarity in the search result is greater than or equal to a first preset threshold; If the similarity is greater than or equal to the first preset threshold, it is determined that there is an identical case set corresponding to the case keyword in the test case set knowledge base, and the target identical case set is obtained and output.
6. The test case set generation method according to claim 5, characterized in that: After the step of detecting whether the similarity in the search result is greater than or equal to the first preset threshold, the method further includes: If the similarity is less than the first preset threshold, detecting whether the similarity is greater than a second preset threshold, wherein the second preset threshold is less than the first preset threshold; If the similarity is greater than the second preset threshold, it is determined that there is a similar case set in the test case set knowledge base, and the steps are executed: obtaining a target similar case set; If the similarity is less than or equal to the second preset threshold, a prompt message is output to prompt the user to manually generate a test case set.
7. The test case set generation method according to claim 6, characterized in that: After the step of outputting prompt information to prompt the user to manually generate a test case set, the method further includes: Obtain a set of annotated test cases manually generated by users; The test case set knowledge base is updated according to the annotated test case set.
8. A test case set generating device, characterized in that: The test case set generating device comprises: An analysis module is used to obtain case keywords, perform semantic analysis on the case keywords, and obtain semantic analysis results; A retrieval module, used to retrieve a test case set knowledge base to obtain a retrieval result according to the semantic analysis result, wherein the test case set knowledge base is generated by training a preset training model obtained by combining a BERT model with a knowledge graph; A first acquisition module, configured to acquire a target similar case set if it is determined that a similar case set exists in the test case set knowledge base according to the search result; The first generation module is used to analyze the case keywords and the target similar case set by using the knowledge graph to generate a test case set.
9. A test case set generating device, characterized in that: The test case set generation device includes: a memory, a processor, and a test case set generation program stored in the memory and executable on the processor. When the test case set generation program is executed by the processor, the steps of the test case set generation method according to any one of claims 1 to 7 are implemented.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a test case set generation program, and when the test case set generation program is executed by a processor, the steps of the test case set generation method according to any one of claims 1 to 7 are implemented.
