Test case completeness evaluation method and device, electronic equipment and storage medium

Abstract

The invention discloses a test case completeness evaluation method and device, electronic equipment and a storage medium. The method comprises the steps that a to-be-evaluated test case is acquired; demand coverage is checked according to the to-be-evaluated test case, a first processing result is obtained, and the first processing result is generated before a test execution stage; entering a test execution stage according to the first processing result; code coverage is checked in the test execution stage to obtain a second processing result, and the second processing result is generated in the process of the test execution stage; and according to the second processing result, realizing test case completeness evaluation. On one hand, demand coverage and code coverage are combined, a test case completeness evaluation method from test case design to test case execution is designed, on the other hand, the test case completeness is supervised in the whole test process, and the test case completeness is guaranteed on the two levels of demand and code.

Description

Technical Field

[0001] The present application relates to the technical field of test case completeness evaluation, and in particular to a test case completeness evaluation method, device, electronic device, and storage medium. Background Art

[0002] The completeness of test case design and execution depends largely on the tester's understanding of the system under test. If the tester's understanding of the system under test is biased or incomplete, it will easily lead to omissions or deficiencies in the test cases, and thus fail to correctly and completely verify the relevant required functions, which will cause certain risks to system safety and quality. Summary of the invention

[0003] The embodiments of the present application provide a test case completeness evaluation method, device, electronic device, and storage medium to organically combine requirement coverage and code coverage, thereby ensuring the completeness of the test to a greater extent and improving the test quality and efficiency.

[0004] The present application embodiment adopts the following technical solutions:

[0005] In a first aspect, an embodiment of the present application provides a test case completeness evaluation method, wherein the method comprises:

[0006] Get the test cases to be evaluated;

[0007] Checking the requirement coverage according to the test case to be evaluated to obtain a first processing result, where the first processing result is generated before the test execution phase;

[0008] Entering the test execution phase according to the first processing result;

[0009] Checking code coverage during the test execution phase to obtain a second processing result, wherein the second processing result is generated during the test execution phase;

[0010] According to the second processing result, the test case completeness evaluation is implemented.

[0011] In some embodiments, checking the requirement coverage according to the test case to be evaluated to obtain a first processing result, wherein the first processing result is generated before the test execution phase, includes:

[0012] Based on natural language processing technology, key elements in the target document are extracted and the requirement document and test case document in the target document are converted into an activity diagram;

[0013] Integrate the nodes and node relationships in the requirement document and the test case document according to the similarity between the texts and abstract the activity diagram into a directed graph; and

[0014] Based on the directed graph, the test paths reflected in the requirement document and the test case document are matched to find out the missing functional points and / or redundant use cases.

[0015] In some embodiments, matching the test paths embodied in the requirement document and the test case document based on the directed graph to find out missing function points and / or redundant use cases includes:

[0016] Based on the directed graph obtained from the requirement document, traverse each directed graph of the test case document and mark the visited nodes and directed edges, wherein the directed graph obtained from each test case in the test case document represents a test path;

[0017] If two or more directed graphs in a test case document have the same nodes and paths visited in the directed graph of the requirement document, the test case will be considered as a redundant test case;

[0018] If after the traversal is completed, there are still nodes and paths in the directed graph of the requirements document that have not been visited, the test case will be regarded as a missing case;

[0019] Output redundant cases based on test case documents and output missing cases based on requirement documents.

[0020] In some embodiments, checking code coverage during the test execution phase to obtain a second processing result, wherein the second processing result is generated during the test execution phase, includes:

[0021] Check code coverage during the test execution phase and build a code-function control mapping table;

[0022] Display the relationship between the functional controls of the solution interface, and combine the coverage information of the functional points;

[0023] The missing test cases are supplemented according to the uncovered function information found in the code-function control mapping table.

[0024] In some embodiments, checking code coverage during the test execution phase and constructing a code-function control mapping table includes:

[0025] Traversing the source program code to determine whether the code has established a mapping relationship in the code-function mapping library;

[0026] If it has been built, it is directly added to the code-function control mapping table.

[0027] Otherwise, identify whether it belongs to a defined function control code;

[0028] If the code belongs to a control class definition code block, the corresponding interface control element is searched, and the function control information is added to the code function mapping library and added to the code-function control mapping table;

[0029] Otherwise, the code belongs to a non-defined control class code block, and it is identified whether the code has an associated functional control;

[0030] If it exists, update the relevant code line number of the corresponding function control in the code function mapping library, and update it to the code-function control mapping table synchronously;

[0031] Otherwise, execute the next line of code until the source code traversal is completed.

[0032] The code-function control mapping table information includes at least one of the following information: a function control identifier, an interface display character, a control type, and a related code line number.

[0033] In some embodiments, the relationship between the functional controls of the display interface may be combined with the function point coverage information, including:

[0034] Traverse the source program code, and if the code belongs to a non-defined control class code block and there is a corresponding relationship between functional controls, perform lexical analysis on the code;

[0035] The code is split into a token list according to the predetermined rules, and then parsed using syntax analysis to convert the token list into a tree structure to generate an abstract syntax tree.

[0036] According to the abstract syntax tree of the control information in the code-function control mapping table, find other controls with associated relationships, generate the corresponding function control relationship diagram until the traversal is completed, and form a software function diagram;

[0037] The relationship between the functional controls includes at least one of the following: a parent-child relationship, a sibling relationship, and a calling relationship.

[0038] In some embodiments, supplementing missing test cases according to uncovered function information found in the code-function control mapping table includes:

[0039] According to the function point information corresponding to the uncovered code in the code-function control mapping table;

[0040] Use the software function diagram output by the software function relationship submodule to find the interaction relationship between missing functions and write new test cases.

[0041] In a second aspect, an embodiment of the present application further provides a test case completeness evaluation device, wherein the device comprises:

[0042] The acquisition module is used to obtain the test cases to be evaluated;

[0043] A requirement coverage module, used to check the requirement coverage according to the test case to be evaluated, and obtain a first processing result, wherein the first processing result is generated before the test execution phase;

[0044] A test execution module, configured to enter a test execution phase according to the first processing result;

[0045] A code coverage module, used for checking code coverage in the test execution phase to obtain a second processing result, wherein the second processing result is generated during the test execution phase;

[0046] An evaluation module is used to implement test case completeness evaluation based on the second processing result.

[0047] In a third aspect, an embodiment of the present application further provides an electronic device, comprising: a processor; and a memory arranged to store computer executable instructions, wherein the executable instructions, when executed, cause the processor to perform the above method.

[0048] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, which stores one or more programs. When the one or more programs are executed by an electronic device including multiple application programs, the electronic device executes the above method.

[0049] At least one of the above technical solutions adopted in the embodiments of the present application can achieve the following beneficial effects: obtaining the test cases to be evaluated, and then checking the requirement coverage according to the test cases to be evaluated to obtain a first processing result, which is generated before the test execution phase. Further enter the test execution phase according to the first processing result. Check the code coverage in the test execution phase to obtain a second processing result, which is generated during the test execution phase, and finally implement the test case completeness evaluation according to the second processing result. Through the above method, the requirement coverage and code coverage are combined, and a test case completeness evaluation method from test case design to test case execution is designed. This method realizes the supervision of the completeness of the test cases in the entire test process, and ensures the completeness of the test cases at both the requirement and code levels.

[0050] The method in the embodiment of the present application combines requirement coverage and code coverage, and designs a test case completeness evaluation system from test case design to test case execution. The system realizes the supervision of test case completeness throughout the entire testing process and ensures the completeness of test cases at both the requirement and code levels.

[0051] The method in the embodiment of the present application proposes a source code block operation, which screens out redundant abstract syntax trees while ensuring that the control relationship information in the coverage code is covered, thereby speeding up the generation process of the software function diagram.

[0052] The method in the embodiment of the present application intuitively represents the interactive relationship between software functions by constructing a software function diagram. Testers can quickly understand the overall architecture between functional controls based on missing functional information, thereby improving the speed of writing test cases. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0054] Figure 1 A flowchart of a test case completeness evaluation method in an embodiment of the present application;

[0055] Figure 2 A flowchart of the requirement coverage check in the embodiment of the present application;

[0056] Figure 3 This is a schematic diagram of the document structuring process in the embodiment of the present application;

[0057] Figure 4 A schematic diagram of the process of constructing a test model in an embodiment of the present application;

[0058] Figure 5 A schematic diagram of the code coverage check process in an embodiment of the present application;

[0059] Figure 6 A schematic diagram of the process of converting code information and function point information in an embodiment of the present application;

[0060] Figure 7 A flowchart of the software functional relationship in the embodiment of the present application;

[0061] Figure 8 A flowchart supplementing the test cases in the embodiments of the present application;

[0062] Fig. 9 This is a schematic diagram of the structure of the test case completeness evaluation device in the embodiment of the present application;

[0063] Fig.10 The overall process intention of the test case completeness evaluation method in the embodiment of the present application;

[0064] Fig.11 This is a schematic diagram of the structure of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0065] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0066] In related technologies, test case completeness evaluation mainly includes the following methods:

[0067] (1) Manual use case design and review relies heavily on the tester’s experience and understanding of the system being tested, which results in uneven quality of test cases.

[0068] (2) For large projects, the huge number of requirements increases the possibility of missing or redundant test points, and manual review is costly and time-consuming.

[0069] (3) If uncovered code is found during the testing process, the tester needs to understand the code logic and supplement the test cases, which places higher demands on the tester's ability and the quality of the developer's comments.

[0070] In view of the above-mentioned deficiencies, the test case completeness evaluation method in the embodiment of the present application organically combines requirement coverage and code coverage to ensure the completeness of the test to a greater extent and improve the test quality and efficiency.

[0071] First, in the use case design phase, natural language analysis technology is used to convert the requirement documents and test cases into directed graphs for matching, to discover omissions or redundancies in the use case design process, and to output missing functional points and redundant use cases.

[0072] Secondly, during the use case execution phase, by mapping the code to corresponding functional points and combining it with white-box testing tools, missed code can be discovered and testers can be prompted to make corresponding supplements. The uncovered code functional points are output, allowing testers to supplement test cases well even if they do not understand the code design logic.

[0073] The test case completeness evaluation method in the embodiment of the present application maps the requirement document-test case-code to achieve an organic combination of requirement coverage and code coverage, thereby better ensuring the completeness of the test.

[0074] The test case completeness evaluation method in the embodiment of the present application uses natural language technology to match the requirement document with the test case, which can quickly discover missing test points and redundant test cases, thereby improving the test quality and efficiency.

[0075] The test case completeness evaluation method in the embodiment of the present application maps the code to functional points, which can help testers quickly find missed functional points and complete the supplement of test cases even if they do not fully understand the code.

[0076] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0077] The present application embodiment provides a test case completeness evaluation method, such as Figure 1 As shown, a flow chart of a test case completeness evaluation method in an embodiment of the present application is provided, and the method at least includes the following steps S110 to S150:

[0078] Step S110, obtaining the test case to be evaluated.

[0079] The test cases to be evaluated are the test cases that need to be evaluated for test case completeness, and are usually obtained by testers submitting designed test case set files.

[0080] Step S120, checking the requirement coverage according to the test case to be evaluated to obtain a first processing result, wherein the first processing result is generated before the test execution phase.

[0081] Based on the matching between the requirement document and the test case, and using natural language analysis technology, the activity diagram elements and related relationships in the requirement document are extracted through semantic analysis, and converted into a directed graph to match the test case designed by the tester. The uncovered or repeatedly covered parts of the test case are analyzed and corrected until there are no omissions or redundancies and the use case execution phase is entered.

[0082] Step S130: Entering the test execution phase according to the first processing result.

[0083] The first processing result is generated before the test execution phase, and the test execution phase is entered according to the first processing result.

[0084] Step S140, checking code coverage during the test execution phase to obtain a second processing result, where the second processing result is generated during the test execution phase.

[0085] By matching source code information with functional controls and using the mapping relationship between the two, the conversion of code information to function point information is achieved, and the missing test cases can be directly supplemented based on the uncovered code information.

[0086] Step S150: Implement test case completeness evaluation according to the second processing result.

[0087] First determine whether the source code has been modified. For the first input, unmodified source code, it can be directly input into the white box testing tool to output all code coverage information. For the modified source code, it is necessary to use the stub technology to specially mark the changed code, and use the white box testing tool to monitor the operation of the changed code after the second compilation.

[0088] Furthermore, after obtaining the code running information, the code information and function point information conversion module is used to obtain the function coverage information. For the uncovered function information, new test cases can be directly added in combination with the interface control relationship diagram.

[0089] The method in the embodiment of the present application arranges and integrates the requirement coverage check and the code coverage check by using a pipeline. The test case designer submits the designed test case set file for requirement coverage check. If the test case does not fully cover the requirements, it will be returned to the test case designer for modification; if the test case fully covers the requirements, it will prompt to enter the test execution phase. During the test execution, the code coverage check process will monitor the execution results throughout the process. After the execution is completed, the code coverage check module will input the uncovered code and corresponding functional points according to the code execution situation to prompt the tester to supplement the relevant use cases.

[0090] Through the above method, the textual requirement documents and test cases are converted into structured directed graphs and path matching is performed to realize the completeness check of test cases from the requirement level. This can help testers quickly find missing functional points when faced with a large number of requirement points and ensure the lower limit of test case completeness.

[0091] Through the above method, the block operation of defined classes and non-defined classes is performed through the source program code. Under the premise of ensuring basic information, the code information is more efficiently converted into function point information. By analyzing the uncovered code during the execution process, testers who do not understand the code structure and meaning can quickly locate the test cases that need to be supplemented, thereby improving the test quality and efficiency.

[0092] In summary, the requirement coverage and code coverage are organically combined to achieve mutual verification among requirement document, code design and test cases, and to implement full-process supervision and feedback on test cases from design to execution, thereby reducing the workload of test case review and improving the quality of test case design and execution.

[0093] In one embodiment of the present application, the requirement coverage is checked according to the test case to be evaluated to obtain a first processing result, which is generated before the test execution phase, and includes: based on natural language processing technology, extracting key elements in the target document and converting the requirement document and test case document in the target document into an activity diagram; based on the similarity between the texts, integrating the nodes and node relationships in the requirement document and the test case document and abstracting the activity diagram into a directed graph; and based on the directed graph, matching the test paths embodied in the requirement document and the test case document to find out missing functional points and / or redundant use cases.

[0094] like Figure 2 As shown in the figure, the process of document structuring is as follows: using natural language processing technology to extract key elements from the text, and converting the requirements document and test case document into an activity diagram. Then, based on the similarity between the texts, the nodes and node relationships in the requirements document and the test case document are integrated and abstracted into a directed graph. Finally, the test paths reflected in the requirements document and the test case document are matched to find missing functional points and redundant use cases.

[0095] It can be understood that natural language processing refers to the technology of interactive communication between machines and natural languages ​​used by humans, which is used for text extraction and text comparison in this patent.

[0096] As you can understand, Word2Ver is a group of related models used to generate word vectors. These models are shallow, two-layer neural networks that are trained to reconstruct linguistic word texts. After the model is trained, it can be used to map each word to a vector to represent the relationship between words.

[0097] It should be noted that the stub technology refers to the special marking of the code statements of interest in the source program code, so that during the code execution process, the execution status of the code of interest can be identified through the special marking.

[0098] In one embodiment of the present application, based on the directed graph, the test paths reflected in the requirement document and the test case document are matched to find out the missing functional points and / or redundant use cases, including: based on the directed graph obtained from the requirement document, traversing each directed graph of the test case document, and marking the nodes and directed edges that have been visited, and the directed graph obtained from each use case in the test case document represents a test path; if in the test case document, two or more directed graphs have the same nodes and paths visited in the directed graph of the requirement document, the test case is regarded as a redundant use case; if after the traversal is completed, there are still nodes and paths in the directed graph of the requirement document that have not been visited, the test case is regarded as a missing use case; redundant use cases are output based on the test case document and missing use cases are output based on the requirement document.

[0099] For specific implementation, please refer to Figure 6 ,The generation of the code-function control mapping table requires traversing the source program code to ,determine whether the code has already built a mapping relationship in the ,code function mapping library. If it has been built, it is ,directly added to the code-function control mapping table. Otherwise, it is ,further identified whether it belongs to a defined function control code.

[0100] If the code belongs to a defined control class code block, the corresponding interface control element is searched, and the functional control information is added to the code function mapping library and added to the code-functional control mapping table. Otherwise, the code belongs to a non-defined control class code block, and further identification is performed to determine whether the code has associated functional controls.

[0101] If it exists, update the relevant code line number of the corresponding function control in the code function mapping library and synchronously update it to the code-function control mapping table, otherwise execute the next line of code until the source code traversal is completed.

[0102] It can be understood that the software function diagram refers to the relationship diagram between various functional controls in the software interface, which reflects the dependency relationship between various controls in the process of realizing software functions.

[0103] The method in the embodiment of the present application proposes a conversion relationship between code information and function point information from the perspective of source program code analysis. By performing block operations on the source program code (defined control class code blocks, non-defined control class code blocks), the function control information in the code-function control mapping table can be newly created and updated during the traversal process, thereby quickly realizing the conversion from code information to function point information.

[0104] In one embodiment of the present application, the code coverage is checked during the test execution phase to obtain a second processing result, which is generated during the test execution phase, including: checking code coverage during the test execution phase to construct a code-function control mapping table; displaying the relationship between interface function controls, which can be combined with function point coverage information; and supplementing missing test cases based on uncovered function information found in the code-function control mapping table.

[0105] Please refer to Figure 5 For source code information, first determine whether the source code has been modified. For the first input and unmodified source code, it can be directly input into the white box test tool to output all code coverage information. For modified source code, it is necessary to use the stub technology to specially mark the changed code. After the second compilation, the white box test tool is used to monitor the running status of the changed code. After obtaining the code running information, the code coverage check uses the code information and function point information to convert to obtain function coverage information. For the uncovered function information, new test cases can be directly added in combination with the interface control relationship diagram.

[0106] The method in the embodiment of the present application matches the requirement document and the test case set from the perspective of requirement document analysis, and utilizes natural language technology to more clearly reflect the correspondence between the requirement document and the test case by converting text information into a directed graph and performing test path matching, thereby discovering missing test points and redundant test cases and realizing automatic review of test cases.

[0107] In one embodiment of the present application, checking code coverage in the test execution phase and constructing a code-function control mapping table include: traversing the source program code to determine whether the code has constructed a mapping relationship in the code-function mapping library; if it has been constructed, directly adding it to the code-function control mapping table, otherwise, identifying whether it belongs to a defined function control code; if the code belongs to a defined control class code block, finding the corresponding interface control element, and adding new function control information in the code function mapping library, and adding it to the code-function control mapping table; otherwise, the code belongs to a non-defined control class code block, and identifying whether the code has an associated function control; if it exists, updating the relevant code line number of the corresponding function control in the code function mapping library, and synchronously updating it to the code-function control mapping table; otherwise, executing the next line of code until the source program code traversal is completed, and the code-function control mapping table information includes at least one of the following information: function control identifier, interface display character, control type, and relevant code line number.

[0108] Please refer to Figure 7, build a code-function control mapping table to realize the conversion of code information to function point information, and its workflow is shown in the figure. The generation of the code-function control mapping table requires traversing the source program code to determine whether the code has built a mapping relationship in the code function mapping library. If it has been built, it is directly added to the code-function control mapping table. Otherwise, it is further identified whether it belongs to the definition function control code. If the code belongs to the definition control class code block, the corresponding interface control element is found, and the function control information is added to the code function mapping library and added to the code-function control mapping table at the same time. Otherwise, the code belongs to the non-definition control class code block, and further identifies whether the code has an associated function control. If so, the relevant code line number of the corresponding function control in the code function mapping library is updated and synchronously updated to the code-function control mapping table. Otherwise, the next line of code is executed until the source program code traversal is completed.

[0109] It should be noted that the code-function control mapping table information includes, but is not limited to, function control identifier, interface display characters, control type, related code line number and other information.

[0110] It can be understood that the code function mapping library refers to a storage method that represents the mapping relationship between code and function through the concept of library, thereby realizing the reusability of the mapping relationship between the two.

[0111] In one embodiment of the present application, the relationship between functional controls of the display interface can be combined with function point coverage information, including: traversing the source program code, if the code belongs to a non-defined control class code block and there is a corresponding relationship between functional controls, the code is lexically analyzed; the code is divided into a tokens list according to a predetermined rule and then parsed using syntax analysis, the tokens list is converted into a tree structure representation, and an abstract syntax tree is generated; based on the control information abstract syntax tree in the code-functional control mapping table, other controls with associated relationships are searched, and a corresponding functional control relationship diagram is generated until the traversal is completed to form a software functional diagram; the relationship between the functional controls includes at least one of the following: parent-child relationship, sibling relationship, and call relationship.

[0112] Please refer to Figure 7, which is used to assist testers to quickly understand the relationship between interface function controls, and can be combined with function point coverage information to quickly supplement new test cases. First, the source code needs to be traversed. If the code belongs to a non-defined control class code block and there is a corresponding relationship between function controls, the code is lexically analyzed, and the code is divided into a tokens list according to predetermined rules. Then, the parsing operation is performed using syntax analysis to convert the tokens list into a tree structure, that is, an abstract syntax tree is generated. According to the control information in the code-function control mapping table, the breadth-first traversal algorithm is used to traverse the abstract syntax tree in turn to find other controls with associated relationships, and generate the corresponding function control relationship diagram until the traversal is completed to form a software function diagram.

[0113] The relationship between functional controls mainly includes parent-child relationship, sibling relationship, calling relationship, etc. Parent-child relationship refers to the hierarchical inclusion relationship in the functional control definition, specifically the relationship between the A control defined in this level and the B control of the next level. The sibling relationship refers to the C control of the next level included in the A control level, specifically the relationship between the B control and the C control. The calling relationship means that the realization of a certain control function needs to call another control to complete.

[0114] It should be noted that a token refers to the smallest indivisible unit in a source code program.

[0115] Lexical analysis refers to scanning and reading code statements, breaking the code into lexical units according to predetermined rules, and storing them in a one-dimensional array.

[0116] Syntax analysis is responsible for converting the one-dimensional array generated by lexical analysis into a tree-like expression with nested elements and grammatical meaning, namely, an abstract syntax tree.

[0117] It can be understood that the abstract syntax tree expresses the syntax structure of the source program code through a tree structure, and each node on the tree represents a structure in the source program code.

[0118] In one embodiment of the present application, according to the uncovered function information found in the code-function control mapping table, the missing test cases are supplemented, including: according to the function point information corresponding to the uncovered code in the code-function control mapping table; searching for the interaction relationship between the missing functions through the software function diagram output by the software function relationship, and writing new test cases.

[0119] Please refer to Figure 8, is to quickly supplement the missing test cases based on the uncovered functional information. The function point information corresponding to the uncovered code is found based on the code-function control mapping table output by converting the code information and the function point information, and the software function diagram output by the software function relationship is used to find the interaction relationship between the missing functions, so as to quickly write new test cases.

[0120] The method in the embodiment of the present application proposes a code-function mapping library to store the mapping relationship between code information and function information, and updates and maintains the mapping library information in real time during the implementation process, thereby realizing the reusability of the code-function mapping relationship.

[0121] like Figure 2 As shown, it is a schematic diagram of the workflow of the requirement coverage check module in the embodiment of the present application. The requirement coverage check module mainly includes three parts: document structuring, test model construction, and test path matching. Its workflow is as follows: the document structuring submodule uses natural language processing technology to extract key elements in the text, and converts the requirement document and the test case document into an activity diagram. The activity diagram is used as the input of the test model construction submodule. According to the similarity between the texts, the nodes and node relationships in the requirement document and the test case document are integrated and abstracted into a directed graph. The test path matching submodule matches the test paths reflected in the requirement document and the test case document, and then finds missing function points and redundant use cases.

[0122] like Figure 3 As shown, it is a flow chart of the document structuring submodule in the embodiment of the present application. The document structuring module is mainly used to structure the requirement documents and test case documents described in natural language. Natural language processing (NLP) is used to extract the basic elements and the relationship between the elements contained in the requirement documents and test case documents (hereinafter referred to as documents) respectively, and convert them into activity diagrams to facilitate the subsequent matching between the requirement documents and the test cases.

[0123] The activity diagram is defined as a ternary array P = {D, T, C}, where:

[0124] D represents the set of various nodes in the activity diagram, mainly including the initial node (Initial), action node (Activity), decision node (Decision), and termination node (Final). D = {Initial, Activity, Decision, Final}

[0125] T represents the set of transition edges, T = {t1, t2...t n}

[0126] C represents the control condition corresponding to the transfer edge, C = {c1, c2...c n}

[0127] Then define G as an execution path, which is expressed as

[0128] Where D1 is the initial node (Initial), D n To terminate the node (Final), it is mainly necessary to identify the node D and the relationship between nodes T and C, and then construct the activity diagram.

[0129] First, process the requirements document:

[0130] (1) Use period, semicolon and colon as delimiters to segment the basic document, and use sentences as basic units to identify the relationship between nodes in the activity diagram through conjunctions. The relationship between nodes mainly includes precedence, postorder and decision-making relationships. Some commonly used conjunctions can help to determine the relationship between sentences, such as if...then..., etc., but there are still many sentences without clear conjunctions. Therefore, in view of this situation, some commonly used conjunctions are first included in the vocabulary, and the relationship between the texts is analyzed first. For the remaining uncommon conjunctions, the subject and behavior in the sentence are abstracted, and uncommon keywords are extracted and included in the vocabulary as templates. The scope of conjunctions is continuously expanded to improve the recognition accuracy. This process can be carried out through machine learning methods to continuously improve the recognition accuracy. For sentences where the conjunctions cannot be identified, they are processed by default according to the postorder relationship.

[0131] (2) Further perform part-of-speech analysis on each sentence to obtain the corresponding node information, take the subject-predicate relationship as the basic node, and extract the noun phrase as the participant. On this basis, other grammatical relationships are included in the analysis, including verb-object relationship, preposition-object relationship, etc., taking the basic node as the recognition starting point, according to the results of grammatical analysis, part-of-speech text segmentation, and text segmentation, continuously transfer and merge related words until there are no related words or punctuation marks are encountered, and then output it as an element.

[0132] (3) By analyzing the nodes and node relationships, they are converted into activity diagrams and used as input items for the next module.

[0133] At the same time, the test case document is processed: each case in the test case document represents a test path, and its input data and operation steps are relatively clear. Therefore, according to the string matching algorithm, the nodes and related node relationships can be extracted in the order of steps to build an activity diagram as the input item of the next module. In order to improve the accuracy of the activity diagram generation, the activity diagram output in this module supports manual modification by relevant personnel to further improve the accuracy of the activity diagram.

[0134] Please refer to Figure 4,Workflow diagram of the test model construction module ,The test model construction module is mainly used to identify the nodes and node relationships ,with the same meanings in the requirement document and the test case document, ,and convert the activity diagram into a directed graph for the subsequent ,comparison of the test paths contained in the requirement document and the ,test case document.

[0135] In order to match the test execution path analyzed from the requirements document with the test case in the test case document, it is necessary to perform text semantic matching on the activity graphs constructed from the two types of documents, uniquely identify the nodes and node relationships that express the same semantics, and then abstract them into a directed graph to identify the same test path. The main steps include: text similarity calculation, node and relationship matching, and directed graph conversion.

[0136] (1) Text similarity calculation

[0137] Since the text information contained in the nodes and node relationships in the activity diagram has been abstracted into short texts with simple structures, the text similarity calculation can be converted into word correlation calculation.

[0138] Suppose the text S has n words, s x ={w x1 , w x2 , …w xn ,},The similarity between two texts can be expressed as the sum of the similarities of the corresponding words:

[0139]

[0140] Where sim(w 1i , w 2i ) represents the similarity between the i-th word in text S1 and the j-th word in text S2. To calculate the word similarity, Word2Ver technology is used to convert each word into a vector by training a neural network model. In space, the closer the word vectors are, the higher the similarity.

[0141] The text information contained in node i in the requirement document is set to S 1i , the text information contained in node j in the test case document is set to S 2j For each node in the requirements document, find the information contained in the test case document node p with the highest similarity. 2p , that is, satisfying:

[0142] sim(S 1i , S 2p )≥sim(S 1i , S 2j )

[0143] (2) Node and node relationship matching

[0144] Set a threshold value X for text similarity. For any node i in the requirement document, if sim(S 1i , S 2p )≥X, the text information contained in the requirement document node i is considered to be basically the same as the test case document node p, and the two are the same node. The node is uniquely identified, where node p represents the node with the highest text similarity to the requirement document node i. If sim(S 1i , S 2p )<X, it is considered that there is no node identical to node i in the test case document, and it is treated as an independent node and uniquely identified.

[0145] The relationship between nodes can be judged by the relationship start node, node relationship text similarity calculation and relationship end node. If a node relationship has the same start node and end node, and its relationship text similarity is greater than the threshold value, it is regarded as the same node relationship and aligned for unique identification, otherwise it exists as an independent relationship.

[0146] (3) Directed graph conversion

[0147] According to the identification of nodes and node relationships, the activity diagram extracted from the document is further abstracted and converted into a directed graph. The nodes and node relationships are processed into a data structure that is easy to operate and compare, and are stored and represented, thereby more clearly representing the test path.

[0148] Testing path matching submodules

[0149] The directed graph obtained for each test case in the test case document represents a test path. Therefore, in this module, based on the directed graph obtained from the requirement document, each directed graph in the test case document is traversed, and the visited nodes and directed edges are marked.

[0150] If two or more directed graphs in the test case document have the same nodes and paths visited in the directed graph of the requirement document, they are considered redundant use cases, the set of use cases is marked, and the document is output for manual judgment.

[0151] If after the traversal is completed, there are still nodes and paths in the directed graph of the requirements document that have not been visited, they are regarded as missed use cases, the rule is marked, and the document is output for manual judgment.

[0152] Ultimately, the module will output two documents, one to output redundant cases based on the test case document, and the other to output missing rules based on the requirement document, thereby prompting testers to adjust the test case set.

[0153] like Figure 5The figure shows the workflow of the code coverage check module. The code coverage check module mainly includes three sub-modules: code information and function point information conversion, software function relationship generation, and test case supplementation. Its workflow is shown in the figure. For source code information, first determine whether the source code has been modified. For the first input and unmodified source code, it can be directly input into the white box test tool to output all code coverage information. For modified source code, it is necessary to use the plug-in technology to specially mark the changed code. After the second compilation, the white box test tool is used to monitor the operation of the changed code. After obtaining the code operation information, the code coverage check module uses the code information and function point information conversion module to obtain the function coverage information. For the uncovered function information, new test cases can be directly supplemented in combination with the interface control relationship diagram.

[0154] like Figure 6 The figure shows the workflow diagram of the code information and function point information conversion submodule. The code information and function point information conversion submodule is the core of the code coverage check module. The main function of this module is to build a code-function control mapping table to realize the conversion from code information to function point information. Its workflow is shown in the figure. The generation of the code-function control mapping table requires traversing the source code to determine whether the code has built a mapping relationship in the code function mapping library. If it has been built, it is directly added to the code-function control mapping table. Otherwise, it is further identified whether it belongs to the definition function control code. If the code belongs to the definition control class code block, the corresponding interface control element is searched, and the function control information is added to the code function mapping library and added to the code-function control mapping table at the same time. Otherwise, the code belongs to the non-definition control class code block, and further identifies whether the code has an associated function control. If it exists, the relevant code line number of the corresponding function control in the code function mapping library is updated, and it is synchronously updated to the code-function control mapping table. Otherwise, the next line of code is executed until the source code traversal is completed.

[0155] The code-function control mapping table information includes function control identifier, interface display characters, control type, related code line number and other information.

[0156] Please refer to Figure 7 , is the software functional relationship sub-module workflow diagram.

[0157] The function of the software function relationship submodule is to assist testers to quickly understand the relationship between interface function controls and to quickly supplement new test cases in combination with function point coverage information. The main workflow of this module is shown in the figure. The software function relationship submodule first needs to traverse the source code. If the code belongs to a non-defined control class code block and there is a corresponding relationship between function controls, the code is lexically analyzed and divided into a tokens list according to predetermined rules. Then, the parsing operation is performed using syntax analysis to convert the tokens list into a tree structure, that is, an abstract syntax tree is generated. According to the control information in the code-function control mapping table, the abstract syntax tree is traversed in turn using the breadth-first traversal algorithm to find other controls with associated relationships and generate the corresponding function control relationship diagram until the traversal is completed to form a software function diagram.

[0158] The relationship between functional controls mainly includes parent-child relationship, sibling relationship, calling relationship, etc. Parent-child relationship refers to the hierarchical inclusion relationship in the functional control definition, specifically the relationship between the A control defined in this level and the B control of the next level. The sibling relationship refers to the C control of the next level included in the A control level, specifically the relationship between the B control and the C control. The calling relationship means that the realization of a certain control function needs to call another control to complete.

[0159] Please refer to Figure 8 , is the workflow diagram of the test case supplement submodule. The function of the test case supplement submodule is to quickly supplement the missing test cases based on the uncovered functional information. Its main workflow is shown in the figure. The test case supplement submodule finds the functional point information corresponding to the uncovered code based on the code-functional control mapping table output by the code information and functional point information conversion submodule, and uses the software functional diagram output by the software functional relationship submodule to find the interaction relationship between the missing functions, so as to quickly write new test cases.

[0160] like Fig. 9 As shown, it is an overall flow chart. The overall workflow summarized in the embodiment of the present application mainly includes two modules: a requirement coverage check module and a code coverage check module.

[0161] The requirement coverage check module is based on the matching between the requirement document and the test case. It uses natural language analysis technology to extract the activity diagram elements and related relationships in the requirement document through semantic analysis, and converts them into a directed graph to match the test case designed by the tester. It analyzes and corrects the parts that are not covered or repeatedly covered by the test case until there are no omissions or redundancies and enters the use case execution phase.

[0162] The function of the code coverage checking module is to match the source code information with the functional control and realize the conversion of code information to function point information through the mapping relationship between the two. It can also directly supplement the missing test cases according to the uncovered code information.

[0163] In the embodiment of the present application, a pipeline is used to arrange and integrate the requirement coverage check module and the code coverage check module. The test case designer submits the designed test case set file to the requirement coverage check module. If the test case does not achieve full coverage of the requirements, it will be returned to the test case designer for modification; if the test case achieves full coverage of the requirements, it will prompt to enter the test execution phase. During the test execution, the code coverage check module will monitor the execution results throughout the process. After the execution is completed, the code coverage check module will input the uncovered code and corresponding functional points according to the code execution situation to prompt the tester to supplement the relevant use cases.

[0164] The requirement coverage check module in the embodiment of the present application converts textual requirement documents and test cases into a structured directed graph and performs path matching, thereby checking the completeness of the test cases from the requirement level. It can help testers quickly find missing functional points when faced with a large number of requirement points, thereby ensuring the lower limit of test case completeness.

[0165] The code coverage check module in the embodiment of the present application performs block operations on defined classes and non-defined classes through source program codes. Under the premise of ensuring basic information, it more efficiently converts code information into function point information. By analyzing uncovered code during execution, it helps testers who do not understand the code structure and meaning to quickly locate test cases that need to be supplemented, thereby improving test quality and efficiency.

[0166] The embodiments of the present application organically combine requirement coverage and code coverage to achieve mutual verification among requirement document, code design and test cases, implement full-process supervision and feedback on test cases from design to execution, reduce the workload of test case review, and improve the quality of test case design and execution.

[0167] The present application also provides a test case completeness evaluation device 1000. Fig.10 As shown, a schematic diagram of the structure of a test case completeness evaluation device in an embodiment of the present application is provided, wherein the test case completeness evaluation device 1000 comprises at least: an acquisition module 1010, a requirement coverage module 1020, a test execution module 1030, a code coverage module 1040 and an evaluation module 1050, wherein:

[0168] In one embodiment of the present application, the acquisition module 1010 is specifically used to: acquire the test case to be evaluated.

[0169] The test cases to be evaluated are the test cases that need to be evaluated for test case completeness, and are usually obtained by testers submitting designed test case set files.

[0170] In one embodiment of the present application, the requirement coverage module 1020 is specifically used to: check the requirement coverage according to the test case to be evaluated to obtain a first processing result, and the first processing result is generated before the test execution phase.

[0171] Based on the matching between the requirement document and the test case, and using natural language analysis technology, the activity diagram elements and related relationships in the requirement document are extracted through semantic analysis, and converted into a directed graph to match the test case designed by the tester. The uncovered or repeatedly covered parts of the test case are analyzed and corrected until there are no omissions or redundancies and the use case execution phase is entered.

[0172] In one embodiment of the present application, the test execution module 1030 is specifically used to: enter the test execution phase according to the first processing result.

[0173] The first processing result is generated before the test execution phase, and the test execution phase is entered according to the first processing result.

[0174] In one embodiment of the present application, the code coverage module 1040 is specifically used to: check the code coverage during the test execution phase to obtain a second processing result, and the second processing result is generated during the test execution phase.

[0175] By matching source code information with functional controls and using the mapping relationship between the two, the conversion of code information to function point information is achieved, and the missing test cases can be directly supplemented based on the uncovered code information.

[0176] In one embodiment of the present application, the evaluation module 1050 is specifically used to: implement test case completeness evaluation according to the second processing result.

[0177] First determine whether the source code has been modified. For the first input, unmodified source code, it can be directly input into the white box testing tool to output all code coverage information. For the modified source code, it is necessary to use the stub technology to specially mark the changed code, and use the white box testing tool to monitor the operation of the changed code after the second compilation.

[0178] Furthermore, after obtaining the code running information, the code information and function point information conversion module is used to obtain the function coverage information. For the uncovered function information, new test cases can be directly added in combination with the interface control relationship diagram.

[0179] In one embodiment of the present application, the requirement coverage module 1020 is also used to

[0180] Based on natural language processing technology, key elements in the target document are extracted and the requirement document and test case document in the target document are converted into an activity diagram;

[0181] Integrate the nodes and node relationships in the requirement document and the test case document according to the similarity between the texts and abstract the activity diagram into a directed graph; and

[0182] Based on the directed graph, the test paths reflected in the requirement document and the test case document are matched to find out the missing functional points and / or redundant use cases.

[0183] In one embodiment of the present application, the requirement coverage module 1020 is also used to

[0184] Based on the directed graph obtained from the requirement document, traverse each directed graph of the test case document and mark the visited nodes and directed edges, wherein the directed graph obtained from each test case in the test case document represents a test path;

[0185] If two or more directed graphs in a test case document have the same nodes and paths visited in the directed graph of the requirement document, the test case will be considered as a redundant test case;

[0186] If after the traversal is completed, there are still nodes and paths in the directed graph of the requirements document that have not been visited, the test case will be regarded as a missing case;

[0187] Output redundant cases based on test case documents and output missing cases based on requirement documents.

[0188] In one embodiment of the present application, the code coverage module 1040 is also used to

[0189] Check code coverage during the test execution phase and build a code-function control mapping table;

[0190] Display the relationship between the functional controls of the solution interface, and combine the coverage information of the functional points;

[0191] The missing test cases are supplemented according to the uncovered function information found in the code-function control mapping table.

[0192] In one embodiment of the present application, the code coverage module 1040 is also used to

[0193] Traversing the source program code to determine whether the code has established a mapping relationship in the code-function mapping library;

[0194] If it has been built, it is directly added to the code-function control mapping table.

[0195] Otherwise, identify whether it belongs to a defined function control code;

[0196] If the code belongs to a control class definition code block, the corresponding interface control element is searched, and the function control information is added to the code function mapping library and added to the code-function control mapping table;

[0197] Otherwise, the code belongs to a non-defined control class code block, and it is identified whether the code has an associated functional control;

[0198] If it exists, update the relevant code line number of the corresponding function control in the code function mapping library, and update it to the code-function control mapping table synchronously;

[0199] Otherwise, execute the next line of code until the source code traversal is completed.

[0200] The code-function control mapping table information includes at least one of the following information: a function control identifier, an interface display character, a control type, and a related code line number.

[0201] In one embodiment of the present application, the code coverage module 1040 is also used to

[0202] Traverse the source program code, and if the code belongs to a non-defined control class code block and there is a corresponding relationship between functional controls, perform lexical analysis on the code;

[0203] The code is split into a token list according to the predetermined rules, and then parsed using syntax analysis to convert the token list into a tree structure to generate an abstract syntax tree.

[0204] According to the abstract syntax tree of the control information in the code-function control mapping table, find other controls with associated relationships, generate the corresponding function control relationship diagram until the traversal is completed, and form a software function diagram;

[0205] The relationship between the functional controls includes at least one of the following: a parent-child relationship, a sibling relationship, and a calling relationship.

[0206] In one embodiment of the present application, the code coverage module 1040 is also used to

[0207] According to the function point information corresponding to the uncovered code in the code-function control mapping table;

[0208] Use the software function diagram output by the software function relationship to find the interaction relationship between missing functions and write new test cases.

[0209] It can be understood that the above-mentioned test case completeness evaluation device can implement each step of the test case completeness evaluation method provided in the aforementioned embodiment, and the relevant explanations about the test case completeness evaluation method are applicable to the test case completeness evaluation device, which will not be repeated here.

[0210] Fig.11 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. Fig.11 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. The memory may include a memory, such as a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. Of course, the electronic device may also include hardware required for other services.

[0211] The processor, network interface and memory can be interconnected through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.11 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0212] The memory is used to store the program. Specifically, the program may include a program code, and the program code includes a computer operation instruction. The memory may include a memory and a non-volatile memory, and provides instructions and data to the processor.

[0213] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming a test case completeness evaluation device at the logical level. The processor executes the program stored in the memory and is specifically used to perform the following operations:

[0214] Get the test cases to be evaluated;

[0215] Checking the requirement coverage according to the test case to be evaluated to obtain a first processing result, where the first processing result is generated before the test execution phase;

[0216] Entering the test execution phase according to the first processing result;

[0217] Checking code coverage during the test execution phase to obtain a second processing result, wherein the second processing result is generated during the test execution phase;

[0218] According to the second processing result, the test case completeness evaluation is implemented.

[0219] The above application Figure 1 The method performed by the test case completeness evaluation device disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0220] The electronic device may also perform Figure 1 A method for executing a test case completeness evaluation device in a test case completeness evaluation device, and realizing the test case completeness evaluation device in Figure 1 The functions of the illustrated embodiment will not be described in detail in the embodiments of the present application.

[0221] The present application also provides a computer-readable storage medium, which stores one or more programs, wherein the one or more programs include instructions, which, when executed by an electronic device including multiple application programs, enable the electronic device to execute Figure 1 The method performed by the test case completeness evaluation device in the embodiment shown is specifically used to perform:

[0222] Get the test cases to be evaluated;

[0223] Checking the requirement coverage according to the test case to be evaluated to obtain a first processing result, where the first processing result is generated before the test execution phase;

[0224] Entering the test execution phase according to the first processing result;

[0225] Checking code coverage during the test execution phase to obtain a second processing result, wherein the second processing result is generated during the test execution phase;

[0226] According to the second processing result, the test case completeness evaluation is implemented.

[0227] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0228] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0229] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0230] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0231] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0232] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0233] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0234] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0235] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0236] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A test case completeness evaluation method, wherein: The method comprises: Get the test cases to be evaluated; Checking the requirement coverage according to the test case to be evaluated to obtain a first processing result, where the first processing result is generated before the test execution phase; Entering the test execution phase according to the first processing result; Checking code coverage during the test execution phase to obtain a second processing result, wherein the second processing result is generated during the test execution phase; According to the second processing result, the test case completeness evaluation is implemented.

2. The method of claim 1, wherein: The step of checking the requirement coverage according to the test case to be evaluated to obtain a first processing result, wherein the first processing result is generated before the test execution phase, includes: Based on natural language processing technology, key elements in the target document are extracted and the requirement document and test case document in the target document are converted into an activity diagram; Integrate the nodes and node relationships in the requirement document and the test case document according to the similarity between the texts and abstract the activity diagram into a directed graph; and Based on the directed graph, the test paths reflected in the requirement document and the test case document are matched to find out the missing functional points and / or redundant use cases.

3. The method of claim 2, wherein: The matching of the test paths in the requirement document and the test case document based on the directed graph to find out the missing function points and / or redundant use cases includes: Based on the directed graph obtained from the requirement document, traverse each directed graph of the test case document and mark the visited nodes and directed edges, wherein the directed graph obtained from each test case in the test case document represents a test path; If two or more directed graphs in a test case document have the same nodes and paths visited in the directed graph of the requirement document, the test case will be considered as a redundant test case; If after the traversal is completed, there are still nodes and paths in the directed graph of the requirements document that have not been visited, the test case will be regarded as a missing test case; Output redundant cases based on test case documents and output missing cases based on requirement documents.

4. The method of claim 1, wherein: The checking of code coverage in the test execution phase to obtain a second processing result, wherein the second processing result is generated during the test execution phase, includes: Check code coverage during the test execution phase and build a code-function control mapping table; Display the relationship between the functional controls of the solution interface, and combine the coverage information of the functional points; The missing test cases are supplemented according to the uncovered function information found in the code-function control mapping table.

5. The method of claim 4, wherein: The checking of code coverage during the test execution phase and building of a code-function control mapping table include: Traversing the source program code to determine whether the code has established a mapping relationship in the code-function mapping library; If it has been built, it is directly added to the code-function control mapping table. Otherwise, identify whether it belongs to a defined function control code; If the code belongs to a control class definition code block, the corresponding interface control element is searched, and the function control information is added to the code function mapping library and added to the code-function control mapping table; Otherwise, the code belongs to a non-defined control class code block, and it is identified whether the code has an associated functional control; If it exists, update the relevant code line number of the corresponding function control in the code function mapping library, and update it to the code-function control mapping table synchronously; Otherwise, execute the next line of code until the source code traversal is completed. The code-function control mapping table information includes at least one of the following information: a function control identifier, an interface display character, a control type, and a related code line number.

6. The method of claim 4, wherein: Displays the relationship between the functional controls of the solution interface, and can be combined with the function point coverage information, including: Traverse the source program code, and if the code belongs to a non-defined control class code block and there is a corresponding relationship between functional controls, perform lexical analysis on the code; The code is split into a token list according to the predetermined rules, and then parsed using syntax analysis to convert the token list into a tree structure to generate an abstract syntax tree. According to the abstract syntax tree of the control information in the code-function control mapping table, find other controls with associated relationships, generate the corresponding function control relationship diagram until the traversal is completed, and form a software function diagram; The relationship between the functional controls includes at least one of the following: a parent-child relationship, a sibling relationship, and a calling relationship.

7. The method of claim 4, wherein: Based on the uncovered functional information found in the code-functional control mapping table, missing test cases are supplemented, including: According to the function point information corresponding to the uncovered code in the code-function control mapping table; Use the software function diagram output by the software function relationship to find the interaction relationship between missing functions and write new test cases.

8. A test case completeness evaluation device, wherein: The device comprises: The acquisition module is used to obtain the test cases to be evaluated; A requirement coverage module, used to check the requirement coverage according to the test case to be evaluated, and obtain a first processing result, wherein the first processing result is generated before the test execution phase; A test execution module, configured to enter a test execution phase according to the first processing result; A code coverage module, used for checking code coverage in the test execution phase to obtain a second processing result, wherein the second processing result is generated during the test execution phase; An evaluation module is used to implement test case completeness evaluation based on the second processing result.

9. An electronic device, comprising: processor; as well as A memory arranged to store computer executable instructions, which when executed cause the processor to perform the method of any one of claims 1 to 7.

10. A computer-readable storage medium storing one or more programs, which, when executed by an electronic device including a plurality of application programs, causes the electronic device to execute any one of the methods of claims 1 to 7.

Images