Software Testing Evaluation Method, Device and Storage Medium

Abstract

The present invention discloses a software test evaluation method, which includes obtaining a coverage report generated by a coverage testing tool based on a source code file, wherein the coverage report includes the covered code in the source code file; obtaining a coverage evaluation report generated by an evaluation model based on the covered code and the code to be covered corresponding to the source code file; and determining an evaluation result according to the coverage evaluation report. The present invention also discloses a software test evaluation device and a storage medium. By setting up an evaluation model, the evaluation model generates a coverage evaluation report based on the covered code and the code to be covered, and an accurate software test coverage evaluation result can be obtained according to the coverage evaluation report, solving the problem that the software test coverage cannot be accurately evaluated due to the lack of a unified quantification standard.

Description

Technical Field

[0001] The present invention relates to the field of computer software, and particularly to a software test evaluation method, device and storage medium. Background Art

[0002] The evaluation of software test sufficiency is a very important node in quality assurance work. Currently, the black-box test coverage rate such as the coverage ratio of requirement monitoring points and the white-box test coverage rates such as line coverage, decision coverage, and code branch coverage are generally used to evaluate the coverage rate. However, in black-box testing, since the sufficiency of the test points themselves is questionable and the coverage rate of system implementation cannot be completed, even if the evaluation test coverage rate reaches 100% under this method, there will still be many undetected problems, and the evaluation results of the same test case may be inconsistent for different people. In addition, in white-box testing, the standards for passing the test coverage rate of different modules or systems are inconsistent, that is, there is no unified quantitative standard to measure whether the coverage rate meets the standard, and due to the lack of a unified quantitative standard, it is difficult to formulate test exit conditions. Based on the problems existing in black-box testing and white-box testing, it is easy to lead to low accuracy in the evaluation of software test coverage rate.

[0003] The above content is only used to assist in understanding the technical solution of the present invention, and does not represent an admission that the above content is prior art. Summary of the Invention

[0004] The main purpose of the present invention is to provide a software test evaluation method, device and storage medium, aiming to solve the problem that the software test coverage rate cannot be accurately evaluated due to the lack of a unified quantitative standard.

[0005] To achieve the above purpose, the present invention provides a software test evaluation method, and the steps of the software test evaluation method include:

[0006] Obtain a coverage report generated by a coverage test tool based on a source code file, where the coverage report includes the covered code in the source code file;

[0007] Obtain a coverage evaluation report generated by an evaluation model based on the covered code and the code to be covered corresponding to the source code file;

[0008] Determine an evaluation result according to the coverage evaluation report.

[0009] Optionally, the manner in which the evaluation model generates a coverage evaluation report based on the covered code and the code to be covered corresponding to the source code file includes:

[0010] Obtain the code to be covered corresponding to the source code file based on a pattern matching algorithm;

[0011] Match the covered code with the code to be covered to determine the target covered code in the covered code that matches the code to be covered;

[0012] Generate the coverage evaluation report based on the target covered code.

[0013] Optionally, the step of generating the coverage evaluation report based on the target covered code includes:

[0014] Mark the target code to be covered corresponding to the target covered code as passed the test;

[0015] Mark the other codes to be covered except the target code to be tested as failed the test;

[0016] Generate the coverage evaluation report from the marked codes to be covered.

[0017] Optionally, the step of determining the evaluation result based on the coverage evaluation report includes:

[0018] Obtain the proportion of the codes to be covered marked as passed the test among all the codes to be covered;

[0019] When the proportion is greater than the preset proportion, determine that the evaluation result is sufficient testing.

[0020] When the proportion is less than or equal to the preset proportion, determine that the evaluation result is insufficient testing;

[0021] Optionally, the step of obtaining the codes to be covered corresponding to the source code file based on the pattern matching algorithm includes:

[0022] Obtain the initial test code pattern;

[0023] Use the pattern matching algorithm to match the initial test code pattern with the source code file to obtain the codes to be covered that match the initial test code pattern.

[0024] Optionally, the initial test code pattern includes at least one of a logical judgment function, a non-virtual function, a non-interface definition function, a non-comment code function, and a non-declaration code function.

[0025] Optionally, after the step of determining the evaluation result based on the coverage evaluation report, it further includes:

[0026] When the evaluation result is passed the test, determine that the current software test meets the test exit conditions.

[0027] Optionally, the method further includes:

[0028] Receive new to-be-covered code annotated by the user for the source code file based on the coverage evaluation report;

[0029] Adjust the initial test code pattern according to the new to-be-covered code, wherein the way of adjusting the initial test code pattern includes at least one of adding a code pattern, deleting a code pattern, and correcting a code pattern;

[0030] Correct the evaluation model based on the adjusted initial test code pattern;

[0031] Obtain a new coverage evaluation report generated by the corrected evaluation model based on the covered code and the to-be-covered code corresponding to the source code file;

[0032] Determine the evaluation result according to the new coverage evaluation report.

[0033] In addition, to achieve the above object, the present invention also provides a software test evaluation device, which includes: a memory, a processor, and a software test evaluation program stored on the memory and executable on the processor. When the software test evaluation program is executed by the processor, the steps of the software test evaluation method as described above are implemented.

[0034] In addition, to achieve the above object, the present invention also provides a storage medium, on which a software test evaluation program is stored. When the software test evaluation program is executed by a processor, the steps of the software test evaluation method as described above are implemented.

[0035] A software test evaluation method, device, and storage medium provided by an embodiment of the present invention obtain a coverage report generated by a coverage test tool based on a source code file, wherein the coverage report includes the covered code in the source code file, and then obtain a coverage evaluation report generated by an evaluation model based on the covered code and the to-be-covered code corresponding to the source code file. Finally, the evaluation result is determined according to the coverage evaluation report. By setting an evaluation model in the embodiment of the present invention, the coverage evaluation report is generated by the evaluation model based on the covered code and the to-be-covered code, and an accurate software test coverage evaluation result can be obtained according to the coverage evaluation report, solving the problem that the software test coverage cannot be accurately evaluated due to the lack of a unified quantization standard. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a schematic structural diagram of a software test evaluation device in a hardware operating environment related to the solution of an embodiment of the present invention;

[0037] Figure 2 It is a schematic flowchart of the first embodiment of the software test evaluation method of the present invention;

[0038] Figure 3 It is a detailed flowchart diagram of step S20 in the second embodiment of the software test evaluation method of the present invention;

[0039] Figure 4 It is a detailed flowchart diagram of step S21 in the first embodiment of the software test evaluation method of the present invention;

[0040] Figure 5 It is a detailed flowchart diagram of step S23 in the third embodiment of the software test evaluation method of the present invention;

[0041] Figure 6 It is a schematic diagram of a coverage evaluation report;

[0042] Figure 7 It is a detailed flowchart diagram of step S30 in the fourth embodiment of the software test evaluation method of the present invention;

[0043] Figure 8 It is a flowchart diagram of the fifth embodiment of the software test evaluation method of the present invention.

[0044] The realization of the object, functional features and advantages of the present invention will be further described with reference to the embodiments and the accompanying drawings. Detailed implementation manners

[0045] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0046] The main solution of the embodiment of the present invention is: obtaining a coverage report generated by a coverage test tool based on a source code file, wherein the coverage report includes the covered code in the source code file; obtaining a coverage evaluation report generated by an evaluation model based on the covered code and the code to be covered corresponding to the source code file; and determining an evaluation result according to the coverage evaluation report.

[0047] As Figure 1 shown, Figure 1 It is a schematic diagram of the terminal structure of the hardware operating environment involved in the solution of the embodiment of the present invention.

[0048] The terminal in the embodiment of the present invention can be a PC, or a terminal device with a display function such as a smart phone, a tablet computer, an e-book reader, a portable computer, etc.

[0049] As Figure 1As shown in the figure, the terminal may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, and a communication bus 1002. Among them, the communication bus 1002 is used to implement connection communication between these components. The user interface 1003 may include a display screen (Display) and an input unit such as a keyboard (Keyboard). Optionally, the user interface 1003 may further include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory or a stable memory (non-volatile memory), such as a disk memory. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0050] Those skilled in the art can understand that Figure 1 the terminal structure shown in the figure does not constitute a limitation on the terminal. It may include more or fewer components than shown in the figure, or combine some components, or have different component arrangements.

[0051] As Figure 1 shown in the figure, the memory 1005, as a computer storage medium, may include an operating system, a network communication module, a user interface module, and a software test and evaluation program.

[0052] In Figure 1 the terminal shown in the figure, the network interface 1004 is mainly used to connect to the background server and communicate with the background server for data; the user interface 1003 is mainly used to connect to the client (user side) and communicate with the client for data; and the processor 1001 may be used to call the software test and evaluation program stored in the memory 1005 and perform the following operations:

[0053] Obtain a coverage report generated by the coverage test tool based on the source code file, where the coverage report includes the covered code in the source code file;

[0054] Obtain a coverage evaluation report generated by the evaluation model based on the covered code and the to-be-covered code corresponding to the source code file;

[0055] Determine the evaluation result according to the coverage evaluation report.

[0056] Further, the processor 1001 may call the software test and evaluation program stored in the memory 1005 and further perform the following operations:

[0057] Obtain the to-be-covered code corresponding to the source code file based on the pattern matching algorithm;

[0058] Match the covered code with the code to be covered to determine the target covered code in the covered code that matches the code to be covered;

[0059] Generate the coverage evaluation report according to the target covered code.

[0060] Further, the processor 1001 may call the software test evaluation program stored in the memory 1005 and further perform the following operations:

[0061] Mark the target code to be covered corresponding to the target covered code as passed the test;

[0062] Mark the other codes to be covered except the target code to be tested as failed the test;

[0063] Generate the coverage evaluation report from the marked codes to be covered.

[0064] Further, the processor 1001 may call the software test evaluation program stored in the memory 1005 and further perform the following operations:

[0065] Obtain the proportion of the codes to be covered marked as passed the test among all the codes to be covered;

[0066] When the proportion is greater than the preset proportion, determine that the evaluation result is sufficient test;

[0067] When the proportion is less than or equal to the preset proportion, determine that the evaluation result is insufficient test.

[0068] Further, the processor 1001 may call the software test evaluation program stored in the memory 1005 and further perform the following operations:

[0069] Obtain the initial test code pattern;

[0070] Use a pattern matching algorithm to match the initial test code pattern with the source code file to obtain the codes to be covered that match the initial test code pattern.

[0071] Further, the processor 1001 may call the software test evaluation program stored in the memory 1005 and further perform the following operations:

[0072] When the evaluation result is passed the test, determine that the current software test meets the test exit conditions.

[0073] Further, the processor 1001 may call the software test evaluation program stored in the memory 1005 and further perform the following operations:

[0074] Receiving new to-be-covered code marked by the user for the source code file based on the coverage evaluation report;

[0075] Adjusting the initial test code pattern according to the new to-be-covered code, wherein the way of adjusting the initial test code pattern includes at least one of adding a code pattern, deleting a code pattern, and correcting a code pattern;

[0076] Correcting the evaluation model based on the adjusted initial test code pattern;

[0077] Obtaining a new coverage evaluation report generated by the corrected evaluation model based on the covered code and the to-be-covered code corresponding to the source code file;

[0078] Determining an evaluation result according to the new coverage evaluation report.

[0079] The first embodiment

[0080] Referring to Figure 2 , the first embodiment of the software test evaluation method of the present invention provides a software test evaluation method, and the software test evaluation method includes:

[0081] Step S10, obtaining a coverage report generated by a coverage test tool based on a source code file, wherein the coverage report includes the covered code in the source code file;

[0082] Step S20, obtaining a coverage evaluation report generated by an evaluation model based on the covered code and the to-be-covered code corresponding to the source code file;

[0083] Step S30, determining an evaluation result according to the coverage evaluation report.

[0084] Code coverage is an important indicator reflecting the coverage degree of test cases for the program under test and is also a reference value for measuring test integrity. Whether it is unit testing, application programming interface testing (API), or functional testing, the program code is ultimately called. After the test, through the code coverage data, it can be known whether the test is sufficient and what the weaknesses of the test are, and then it can guide developers or testers to design test cases that can increase the coverage.

[0085] In this embodiment, the coverage testing tool may be selected as the Jacoco tool, which is an open-source coverage tool. The Jacoco tool can be embedded into Ant and MavenAnt, and provides the Ec1Emma Eclipse plugin. It can also use the JavaAgent technology to monitor Java programs. Many third-party tools provide integrations with Jacoco. For example, sonar, Jenkins, etc. It adopts the standard JVMTool Interface. After adding the Jacoco Agent to the JVM and starting it, the specific code lines executed in the classes loaded into the JVM will be recorded. Jacoco collects this execution information and generates a coverage report after the JVW ends. The jacoco tool includes coverage counters at multiple scales, including instruction-level coverage (Instructions, COcoverage), branches (Branches, C1coverage), cyclomatic complexity (CyclomaticComplexity), line coverage (Lines), method coverage (non-abstract methods), and class coverage (classes).

[0086] Optionally, the coverage report includes code coverage, and the code coverage includes at least one of the following: instruction coverage, branch coverage, line coverage, method coverage, and class coverage. The present invention is not limited thereto.

[0087] Optionally, the coverage report further includes the code that has been executed in the source code file, that is, the covered code covered in the test case. The covered code can be a covered branch, a covered instruction, or a covered code line. In this embodiment of the application, the covered code is taken as an example of a covered code line for analysis.

[0088] Optionally, after obtaining the coverage report, a coverage result view corresponding to the source code file is generated according to the coverage report. The coverage result view displays the covered code and the non-covered code other than the covered code in different display manners. Specifically, different color backgrounds can be added to different codes in the source code file. Red background: no coverage, none of the instructions in this line are executed; yellow background: partial coverage, some instructions in this line are executed; green background: full coverage, all instructions in this line are executed. It can be understood that displaying the coverage result view can quickly enable testers to understand the code coverage situation.

[0089] Optionally, in another embodiment, during the process of generating the coverage report, the covered code can also be highlighted in the coverage report. The number of covered lines of code under a certain package, class, or method can be calculated separately, as well as the total number of covered lines of code. Testers can identify the code locations in the program that have not been executed based on the coverage rate, so as to modify the code and design more effective test cases for the coverage rate to improve the code coverage rate. This not only improves the accuracy of obtaining the code coverage rate but also enables testers to more accurately know the accuracy of the test, improving the convenience of testers for code testing.

[0090] Optionally, after generating the coverage report, the coverage report and the source code file are input into an evaluation model. The evaluation model is used to generate a coverage evaluation report based on the coverage report and the source code file. Specifically, the code coverage situation of each piece of code to be covered is determined according to the covered code in the coverage report and the code to be covered in the source program file, and the coverage evaluation report is generated based on the code coverage situation. That is, the coverage evaluation report includes the code coverage situation of the code to be covered, and the code coverage situation includes covered and not covered. When the code to be covered exists in the covered code, it means the code to be covered has been covered; when the code to be covered does not exist in the covered code, it means the code to be covered has not been covered. Among them, the code to be covered can be selected by experts according to the current test requirements, or obtained using a pattern matching algorithm. The code to be covered can be all lines of code in the source program file or part of the lines of code in the source program file, and the code to be covered is used to represent the lines of code that need to be covered in the current software test.

[0091] Optionally, the coverage evaluation report also includes the coverage rate of the code to be covered. The coverage rate is the ratio of the covered code to be covered to the code to be covered. Among them, the covered code to be covered includes at least one line of code in the covered code. Specifically, the covered code and the code to be covered are matched to obtain the code to be covered that matches the covered code, and the code to be covered that matches the covered code is determined as the covered code to be covered. Then, according to the number of lines of the covered code to be covered and the number of lines of the code to be covered, the ratio of the covered code to be covered to the code to be covered is determined, and the coverage evaluation report is generated based on the ratio.

[0092] Optionally, the coverage evaluation report may further include the similarity between the covered code and the code to be covered. Specifically, the covered code of the current software test is matched with the code to be covered to obtain the similarity between the covered code and the code to be covered, and a coverage evaluation report is generated based on the similarity. It can be understood that if the similarity between the covered code and the code to be covered is greater than or equal to a preset threshold, it means that all the code to be covered is included in the covered code. If the similarity between the covered code and the code to be covered is less than the preset threshold, it means that there is code in the code to be covered that has not been covered. The preset threshold can be 100%, or it can be set by the tester according to their testing experience.

[0093] Optionally, after obtaining the coverage evaluation report, the evaluation result is determined based on the coverage evaluation report. The evaluation result includes the sufficiency result of the current software test, where the sufficiency result includes sufficient testing and insufficient testing.

[0094] Optionally, after generating the coverage evaluation report, based on the code coverage situation of the code to be covered included in the coverage evaluation report, the way to generate the evaluation result can be: judging whether each code to be covered is covered according to the code coverage situation of each code to be covered. If all the codes to be covered are covered and the proportion of the covered codes to be covered in the codes to be covered is 100%, then the evaluation result is determined to be sufficient testing; if there are codes to be covered that are not covered and the proportion of the covered codes to be covered in the codes to be covered is less than 100%, then the evaluation result is determined to be insufficient testing, and the tester can modify the test cases according to the evaluation result.

[0095] Optionally, based on the coverage evaluation report, the coverage rate of the code to be covered may further be included. Another way to generate the evaluation result based on the coverage report can be to compare whether the coverage rate is greater than a preset coverage rate. If the coverage rate is greater than or equal to the preset coverage rate, then the evaluation result is determined to be sufficient testing; if the coverage rate is less than the preset coverage rate, then the evaluation result is determined to be insufficient testing.

[0096] Optionally, the evaluation result further includes whether the test exit condition is met. The test exit condition may be that the evaluation result is sufficient testing. Specifically, if the sufficiency result of the current software test is sufficient testing, it is determined that the test exit condition is met, and then the software test can be stopped; when the sufficiency result of the current software test is insufficient testing, it is determined that the test exit condition is not met, and the tester modifies the test cases according to the evaluation result until the sufficiency result of the current software test is sufficient testing and the test exit condition is met.

[0097] In the embodiments of the present application, by setting up an evaluation model, the to-be-covered code that should be executed in the source program file is determined according to the evaluation model, and the to-be-covered code and the covered code that is actually executed are matched to determine the code coverage of each to-be-covered code, and the coverage rate evaluation report can be generated according to the code coverage. Based on the coverage rate evaluation report, the user can determine the evaluation result of the current software test. The evaluation result includes the sufficiency result of the current software test and whether the test exit condition is met. When it is determined according to the coverage rate evaluation report that the code coverage of each to-be-covered code is covered, it can be determined that all the to-be-covered code that should be executed is covered, and then it can be determined that the coverage rate of the current software test has been marked, and further it can be determined that the current software test is sufficient and meets the test exit condition. In the embodiments of the present application, the code coverage of the to-be-covered code is used as a quantitative standard for quantifying the sufficiency of software testing, which improves the accuracy of software test coverage rate evaluation.

[0098] Second Embodiment

[0099] Optionally, referring to Figure 3 , based on the first embodiment, the manner in which the evaluation model generates a coverage rate evaluation report based on the covered code and the to-be-covered code corresponding to the source code file includes:

[0100] S21, obtaining the to-be-covered code corresponding to the source code file based on a pattern matching algorithm;

[0101] S22, matching the covered code with the to-be-covered code to determine the target covered code in the covered code that matches the to-be-covered code;

[0102] S23, generating the coverage rate evaluation report according to the target covered code.

[0103] Optionally, referring to Figure 4 , the manner of obtaining the to-be-covered code corresponding to the source code file based on a pattern matching algorithm is:

[0104] S211, obtaining an initial test code pattern;

[0105] S212, using the pattern matching algorithm to match the initial test code pattern with the source code file to obtain the to-be-covered code that matches the initial test code pattern.

[0106] Optionally, the initial test code pattern includes at least one of, but is not limited to, a logical judgment function, a non-virtual function, a non-interface definition function, a non-comment code function, and a non-declaration code function. The initial test code pattern can be preset by the tester according to the test requirements, or can be adjusted in real time by the tester according to the software test situation. The initial test code pattern is used to represent the type of code to be covered.

[0107] Optionally, after determining the initial test code pattern, a pattern matching algorithm is used to match the initial test code pattern with the source code file to obtain the code to be covered that matches the initial test code pattern, that is, the code of the source program file that matches the initial test code pattern.

[0108] Optionally, in another embodiment, the method for obtaining the code to be covered can also be to receive the target code confirmed by the user as the code to be covered, and determine the target code as the code to be covered. Optionally, after receiving the target code confirmed by the user as the code to be covered, the code pattern corresponding to the target code is determined according to the target code, and the initial test code pattern is adjusted according to the code pattern to improve the evaluation model.

[0109] Optionally, after obtaining the code to be covered, the covered code is matched with the code to be covered to determine the target covered code in the covered code that matches the code to be covered. It can be understood that the code to be covered includes the target covered code, and the covered code includes the target covered code.

[0110] Optionally, the method of matching the covered code with the code to be covered can be to compare each line of the covered code with the code to be covered one by one.

[0111] Optionally, after determining the target covered code in the covered code that matches the code to be covered, a coverage evaluation report is generated according to the target covered code. The coverage evaluation includes the code coverage situation for each code to be covered. The code coverage situation includes covered and not covered, which is used to represent whether the code to be covered is covered. It can be understood that when the code to be covered has been covered, it means that there is a target covered code in the covered code that matches the code to be covered, that is, the current software test has executed the code to be covered; when the code to be covered has not been covered, it means that there is no target covered code in the covered code that matches the code to be covered, that is, the current software test has not executed the code to be covered.

[0112] Optionally, the code coverage situation can be identified by marking the code to be covered. Different code coverage situations correspond to different marking methods. For example, the code to be covered that has been covered can be marked as covered, and the code to be covered that has not been covered can be marked as not covered. Also, the background color of the code to be covered that has been covered can be marked as green, and the background color of the code to be covered that has not been covered can be marked as red. Additionally, the code coverage situation can be identified by retaining or deleting the code to be covered. When the code to be covered has been covered, the covered code to be covered is deleted. When the code to be covered has not been covered, the uncovered code to be covered is retained, so that testers can modify the code based on the retained code to be covered and design more effective test cases for coverage to improve the code coverage. This not only improves the accuracy of obtaining the code coverage, but also allows testers to more accurately know the accuracy of the test, improving the convenience of testers for code testing.

[0113] Third Embodiment

[0114] Based on the above embodiments, in order to allow testers to quickly understand the coverage situation of the code to be covered, an embodiment of the present application also proposes a method for identifying the code coverage situation. Referring to Figure 5 , the step S23 includes:

[0115] S231, marking the target code to be covered corresponding to the target covered code as passed the test;

[0116] S232, marking the other codes to be covered except the target code to be tested as not passed the test;

[0117] S233, generating the coverage evaluation report from the marked codes to be covered.

[0118] Optionally, the code to be covered corresponding to the target covered code is marked as passed the test, and the code to be covered that does not match the target covered code is marked as not passed the test. Referring to Figure 6 , Figure 5 shows a schematic diagram of marking the code to be covered.

[0119] Optionally, after completing the marking of each code to be covered, the marked codes to be covered are used to generate the coverage evaluation report.

[0120] Optionally, the coverage evaluation report may further include the coverage sufficiency results of each code block. It can be understood that a code block includes at least one line of code, the code to be covered includes multiple code blocks, and each code block includes at least one corresponding line of code. When all lines of code in a code block are marked as passed the test, it means that the code block has been covered, that is, the code block has been tested sufficiently. When there is a line of code marked as failed the test among the lines of code in a code block, it means that the code block is not fully covered or not covered at all, that is, the code block has not been tested sufficiently.

[0121] Optionally, the coverage sufficiency result of the code block may include the number of lines of code that passed the test and the number of lines of code that failed the test. Refer to Figure 6 , Figure 6 The "test coverage sufficiency result: 2 passed, 1 failed" shown is used to represent the code coverage situation and the coverage sufficiency result of the current code block.

[0122] In the embodiments of the present application, based on marking the code coverage situation of each piece of code to be covered in the coverage evaluation report, the user can quickly determine which pieces of code to be covered have been executed and which pieces of code to be covered have not been executed based on the coverage evaluation report. Based on the marked code to be covered, the code coverage situation can be quickly understood, and based on the code coverage situation, the sufficiency of the current software test can be quickly evaluated. It can be understood that if all the code to be covered is covered, it is determined that the coverage rate of the current software test meets the standard and the software test is sufficient, improving the accuracy of the software test sufficiency evaluation.

[0123] Fourth Embodiment

[0124] Optionally, based on the third embodiment, the embodiments of the present application propose a method for determining the evaluation result. Refer to Figure 7 , S30 includes:

[0125] Step S31, obtaining the proportion of the code to be covered marked as passed the test in all the code to be covered;

[0126] Step S32, when the proportion is greater than the preset proportion, determining that the evaluation result is test sufficient;

[0127] Step S33, when the proportion is less than or equal to the preset proportion, determining that the evaluation result is test insufficient.

[0128] In the embodiments of the present application, after generating the coverage evaluation report, based on the code coverage of each code to be covered and the coverage sufficiency results of each code block included in the coverage evaluation report, testers can quickly determine the evaluation result of the current software test based on the coverage evaluation report. The evaluation result is used to indicate whether the current software test is insufficient. The evaluation result includes the evaluation results of sufficient testing or insufficient testing. Specifically, it can be determined whether each code to be covered is covered according to the coverage evaluation report. When each code to be covered is covered, that is, when each code to be covered exists in the covered code, it means that the code blocks corresponding to each code to be covered are sufficiently tested, that is, it means that the current software test has been sufficiently tested, and the evaluation result is sufficient testing; when there are codes to be covered that are not covered among each code to be covered, that is, when each code to be covered includes codes that do not match the covered code, it means that the code blocks corresponding to each code to be covered have not been sufficiently tested, that is, it means that the current software test has not been sufficiently tested, and the evaluation result is insufficient testing.

[0129] Optionally, the method of determining whether each code to be covered is covered can be to obtain the proportion of the code to be covered marked as passed the test among all the codes to be covered. Specifically, the number of lines of the code to be covered marked as passed the test and the number of lines of the code to be covered can be obtained, and the proportion corresponding to these two numbers of lines is determined as the proportion of the code to be covered marked as passed the test among all the codes to be covered; in addition, the number of lines of the code to be covered marked as failed the test and the number of lines of the code to be covered can also be obtained, and the difference between the proportion corresponding to these two numbers of lines and the preset proportion is determined as the proportion of the code to be covered marked as passed the test among all the codes to be covered, and the preset proportion is 100%.

[0130] Optionally, after determining the proportion, the proportion is compared with the preset proportion. When the proportion is greater than the preset proportion, it is determined that the evaluation result is sufficient testing; when the proportion is less than or equal to the preset proportion, it is determined that the evaluation result is insufficient testing. The preset proportion is formulated by testers according to test experience, which can be 100%, or 98%, or 99%. The embodiments of the present application do not make any limitations.

[0131] Optionally, when it is determined that the evaluation result is sufficient testing, it means that the current software test coverage rate has reached the standard, it is determined that the current software test meets the test exit conditions, and then the software test is exited. Among them, the test exit condition is that the evaluation result is sufficient testing.

[0132] In the embodiment of the present application, after obtaining the coverage rate evaluation report, the evaluation result is determined according to the annotation status of each piece of code to be covered in the coverage rate evaluation report. The code to be covered marked as passing the test has been actually covered, and the code to be covered marked as failing the test has not been covered. According to the proportion of the code to be covered marked as passing the test in all the code to be covered, the sufficiency result of the current software test can be determined. When the proportion is greater than the preset proportion, it is determined that all the code to be covered has been actually covered, that is, it can be determined that the current software test is sufficient, and the evaluation result can be determined as test sufficient. When the proportion is less than the preset proportion, it is determined that there is code among the code to be covered that has not been covered, that is, it can be determined that the current software test is insufficient, and the evaluation result can be determined as test insufficient. In the embodiment of the present application, the proportion corresponding to the code to be covered passing the test is used as a quantitative standard for quantifying whether the test is sufficient, which can not only specify the test exit condition, but also improve the accuracy of software test evaluation.

[0133] Fifth Embodiment

[0134] Based on all the above embodiments, referring to Figure 8 , the method further includes:

[0135] Step S40: Receive new code to be covered marked by the user for the source code file based on the coverage rate evaluation report;

[0136] Step S50: Adjust the initial test code pattern according to the new code to be covered, wherein the way of adjusting the initial test code pattern includes at least one of adding a code pattern, deleting a code pattern, and correcting a code pattern;

[0137] Step S60: Modify the evaluation model based on the adjusted initial test code pattern;

[0138] Step S70: Obtain a new coverage rate evaluation report generated by the modified evaluation model based on the covered code and the code to be covered corresponding to the source code file;

[0139] Step S80: Determine the evaluation result according to the new coverage rate evaluation report.

[0140] In an embodiment of the present application, after generating the coverage rate evaluation report, the coverage rate evaluation report includes the code coverage of each code to be covered and / or the coverage sufficiency results of each code block. The user independently determines whether the coverage rate evaluation report fully meets the test requirements based on the coverage rate report and the test requirements. When the test requirements are met, the test is exited. When the test requirements are not met, the tester can select code lines in the source code file to select new code to be covered. Optionally, the way of selecting code lines in the source code file can receive the annotation of a certain code line or code block in the source program file, determine the annotated code line or code block as the code line selected by the user, and determine the annotated code line or code block as the new code to be covered. It can be understood that the new code to be covered is different from the code to be covered generated by using the evaluation model.

[0141] Optionally, after determining the new code to be covered, adjust the initial test code pattern according to the new code to be covered. Among them, the way of adjusting the initial test code pattern includes at least one of adding a code pattern, deleting a code pattern, and correcting a code pattern. Specifically, a new code pattern can be determined according to the new code to be covered, and the initial test code pattern can be adjusted according to the new code pattern.

[0142] Optionally, after adjusting the initial test code pattern, correct the evaluation model based on the adjusted initial test code model, generate a new coverage rate evaluation report according to the corrected evaluation model, and determine the evaluation result according to the new coverage rate report. Specifically, determine the code to be covered according to the adjusted initial test code pattern in the corrected evaluation model. The code to be covered includes the new code to be covered, and may also include the code to be covered other than the new code to be covered and corresponding to the new code model. Match the code to be covered with the covered code to determine the target covered code in the covered code that matches the code to be covered. Then mark the target code to be covered corresponding to the target covered code as passed the test and mark the other codes to be covered except the target code to be tested as not passed the test. Then generate the new coverage rate evaluation report with the marked codes to be covered, and determine the evaluation result according to the new coverage rate evaluation report. It can be understood that after the evaluation model is corrected, the corrected evaluation model can be used not only in the sufficiency evaluation of the current software test, but also in subsequent software tests. Based on the continuous correction of the evaluation model, the higher the accuracy of the evaluation model in determining the code to be covered, the higher the accuracy of generating the coverage rate evaluation report.

[0143] Optionally, in another embodiment, the way to generate the evaluation result may also be that after receiving the new code to be covered marked by the user based on the coverage evaluation report for the source code file, directly match the new code to be covered with the covered code in the coverage report. If the covered code includes the new code to be covered, determine that the new code to be covered passes the test, mark the new code to be covered as passed the test, adjust the coverage evaluation report according to the marked new code to be covered to generate a new coverage evaluation report, and determine the evaluation result according to the new coverage evaluation report.

[0144] In the embodiments of the present application, after generating the coverage evaluation report, by receiving the new code to be covered marked by the user, determine the new code pattern corresponding to the new code to be covered according to the new code to be covered, adjust the initial test code pattern according to the new code model, and correct the evaluation model according to the adjusted initial test code model, so that the corrected evaluation model continues to process the coverage report of the current software test. In the embodiments of the present application, by continuously adjusting the initial test code pattern, sufficient evaluation experience is accumulated, and an algorithm expert generates and optimizes the evaluation model according to the adjusted initial test code pattern, which can not only automatically evaluate the test sufficiency, but also obtain an accurate coverage evaluation report and improve the evaluation accuracy of the coverage sufficiency of the current software test.

[0145] In addition, an embodiment of the present invention further provides a storage medium, on which a software test evaluation program is stored. When the software test evaluation program is executed by a processor, the steps of the above-mentioned various embodiments are implemented.

[0146] It should be noted that in this article, the term "including", "comprising" or any other variant thereof is intended to cover a non-exclusive inclusion, so that a process, method, article or system including a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or system. Without further limitation, an element defined by the statement "including a..." does not exclude the existence of additional identical elements in the process, method, article or system including the element.

[0147] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages and disadvantages of the embodiments.

[0148] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-described embodiment methods can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above and includes several instructions for causing a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in various embodiments of the present invention.

[0149] The above are only the preferred embodiments of the present invention, and do not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by using the specification and drawings of the present invention, or directly or indirectly applied in other related technical fields, shall be equally included in the patent protection scope of the present invention.

Claims

1. A software testing and evaluation method, characterized in that, The steps of the software test evaluation method include: Obtain a coverage report generated by a coverage testing tool based on a source code file, where the coverage report includes the covered code in the source code file; Input the coverage report and the source code file into an evaluation model; Obtain a coverage evaluation report generated by the evaluation model based on the covered code and the code to be covered corresponding to the source code file; Use a pattern matching algorithm to match an initial test code pattern with the source code file, and obtain the code to be covered that matches the initial test code pattern. The initial test code pattern includes at least one of, but is not limited to, a logical judgment function, a non-virtual function, a non-interface definition function, a non-comment code function, and a non-declaration code function; Compare each line of the covered code with the code to be covered one by one to determine the target covered code in the covered code that matches the code to be covered; Mark the code to be covered corresponding to the target covered code as passed the test; Mark the code to be covered that does not match the target covered code as failed the test; Generate the coverage evaluation report from the marked code to be covered; Determine the evaluation result according to the coverage evaluation report.

2. The method according to claim 1, characterized in that, The step of determining the evaluation result according to the coverage evaluation report includes: Obtain the proportion of the code to be covered marked as passed the test among all the code to be covered; When the proportion is greater than a preset proportion, determine that the evaluation result is sufficient testing; When the proportion is less than or equal to the preset proportion, determine that the evaluation result is insufficient testing.

3. The method according to any one of claims 2, characterized in that, After the step of determining that the evaluation result is insufficient testing when the proportion is less than or equal to the preset proportion, it further includes: When the evaluation result is sufficient testing, determine that the current software test meets the test exit condition.

4. The method according to any one of claims 1 to 3, characterized in that The method further includes: Receive new code to be covered marked by the user for the source code file based on the coverage evaluation report; Adjust the initial test code pattern according to the new code to be covered, where the way of adjusting the initial test code pattern includes at least one of adding a code pattern, deleting a code pattern, and correcting a code pattern; Modify the evaluation model based on the adjusted initial test code pattern; Obtain a new coverage evaluation report generated by the modified evaluation model based on the covered code and the code to be covered corresponding to the source code file; Determine the evaluation result according to the new coverage evaluation report.

5. A software test evaluation device, characterized in that, The software test evaluation device includes: a memory, a processor, and a software test evaluation program stored on the memory and executable on the processor. When the software test evaluation program is executed by the processor, it implements the steps of the software test evaluation method according to any one of claims 1 to 4.

6. A storage medium, characterized in that, A software test evaluation program is stored on a storage medium. When the software test evaluation program is executed by a processor, it implements the steps of the software test evaluation method according to any one of claims 1 to 4.

Images