A software testing method, apparatus, device, and medium

Abstract

This application discloses a software testing method, apparatus, device, and medium, relating to the field of software testing technology. The method includes: determining the software test cases to be evaluated and identifying several corresponding source code files; parsing each source code file to obtain code comments, and parsing the software test cases to obtain relevant case information, and saving this information to a preset case information list; performing a quality assessment on the code comments; if the quality assessment passes, obtaining code-related information from several source code files based on the code comments and saving this information to a preset code information list; comparing and matching the information in the preset code information list and the preset case information list using preset natural language processing technology, determining whether the coverage of the software test cases meets preset coverage conditions based on the matching results, and performing software testing operations on several source code files based on the software test cases after determining that the preset coverage conditions are met.

Description

Technical Field

[0001] This invention relates to the field of software testing technology, and in particular to a software testing method, apparatus, equipment, and medium. Background Technology

[0002] In software development, developers write code based on requirements documents to implement relevant functions, and testers similarly write test cases based on the requirements documents to test the submitted functions. Since testers cannot know the effectiveness of the code, there is no guarantee before testing whether their test cases fully cover the functions involved. Therefore, test case coverage assessment is necessary. Traditionally, ensuring test case coverage relies on manual software testing followed by coverage assessment. One approach is to analyze the number of bugs (vulnerabilities) to assess development quality and decide whether to strengthen testing, which can lead to repeated testing and extend the development cycle. Another approach is to judge the adequacy of testing based on bug escape rates, which can increase production failure rates and lead to irreversible production accidents, resulting in low software testing efficiency.

[0003] In conclusion, optimizing the coverage evaluation process for software test cases to improve the efficiency of software testing is a pressing technical issue that needs to be addressed. Summary of the Invention

[0004] In view of this, the purpose of this invention is to provide a software testing method, apparatus, device, and medium that can optimize the coverage evaluation process of software test cases to improve the efficiency of software testing. The specific solution is as follows:

[0005] Firstly, this application provides a software testing method, including:

[0006] Identify the software test cases to be evaluated and identify several source code files corresponding to the software test cases; the source code files are pre-developed software source code files with code comments based on a preset annotation setting format; the preset annotation setting format includes preset custom annotations;

[0007] The source code files are parsed to obtain code comments for the development code in the source code files, and the software test cases are parsed to obtain corresponding case-related information, and the case-related information is saved to a preset case information list;

[0008] A quality assessment is performed on the code comments. If the assessment result indicates that the code comments have passed the quality assessment, then the code-related information of the several source code files is obtained based on the code comments, and the code-related information is saved to a preset code information list.

[0009] Based on the preset natural language processing technology, the information in the preset code information list and the preset case information list is compared and matched to determine whether the coverage of the software test case meets the preset coverage condition. After determining that the coverage meets the preset coverage condition, software testing operations are performed on the several source code files based on the software test case.

[0010] Each of the aforementioned related information includes corresponding functional module information and logical detail information; the preset case information list includes the first functional module list and the first logical detail list of the software test cases; the preset code information list includes the second functional module list and the second logical detail list of the plurality of source code files.

[0011] Optionally, determining the current software test case to be evaluated and determining the corresponding source code files for the software test case includes:

[0012] Identify the software test cases to be evaluated and their corresponding target code requirement numbers;

[0013] Based on the target code requirement number, several source code files corresponding to the software test case are determined; the several source code files are source code files whose file names contain code requirement numbers that match the target code requirement number.

[0014] Optionally, the step of parsing the plurality of source code files to obtain code comments for the development code in the plurality of source code files, parsing the software test cases to obtain corresponding case-related information, and saving the case-related information to a preset case information list includes:

[0015] The preset custom annotations are identified in the development code of the aforementioned source code files respectively, so as to read the code comments of the development code based on the obtained identification results;

[0016] Read the functional module field of each record in the body of the software test case to obtain the first functional module information;

[0017] The test scenario fields of each record in the table body are read, and semantic analysis is performed on the test scenario fields based on a preset language model to obtain the first logical detail information based on the analysis results; the case-related information includes the first functional module information and the first logical detail information.

[0018] The first functional module information is saved to the first functional module list, and the first logical detail information is saved to the first logical detail list;

[0019] Accordingly, the step of obtaining code-related information of the plurality of source code files based on the code comments and saving the code-related information to a preset code information list includes:

[0020] The second functional module information and the second logical detail information in the code comments are identified according to the preset annotation setting format; the code-related information includes the second functional module information and the second logical detail information;

[0021] The second functional module information is saved to the second functional module list, and the second logical detail information is saved to the second logical detail list.

[0022] Optionally, the quality assessment of the code comments includes:

[0023] Determine whether target description keywords exist in the code comments; the target description keywords include first description keywords representing functional descriptions and second description keywords representing logical descriptions;

[0024] If the target description keyword exists in the code comments, the number of times the second description keyword appears in the code comments is counted, and the total number of lines of code in the plurality of source code files is determined;

[0025] A target occurrence frequency threshold is determined based on the total number of lines of code, and the occurrence frequency is compared with the target occurrence frequency threshold to obtain the corresponding comparison result;

[0026] If the comparison result indicates that the occurrence frequency is not lower than the target occurrence frequency threshold, then an evaluation result indicating that the code annotation has passed the quality assessment is obtained.

[0027] Optionally, the step of comparing and matching information in the preset code information list and the preset case information list based on preset natural language processing technology includes:

[0028] Determine the number of first elements stored in the first target list and the number of second elements stored in the corresponding second target list; wherein, if the first target list is the first functional module list, then the corresponding second target list is the second functional module list; if the first target list is the first logical detail list, then the corresponding second target list is the second logical detail list.

[0029] If the number of the first element is the same as the number of the second element, the content text of the first item in the first target list is determined as the target matching data, and based on the preset natural language processing technology, the target matching data is compared and matched with the content text of all items in the second target list in turn to obtain the corresponding matching result;

[0030] If the matching result indicates that there is content text in the second target list that matches the target matching data, then the content text of the next item in the first target list is determined as the latest target matching data, and the process jumps to the step of comparing and matching the target matching data with the content text of all items in the second target list in turn based on preset natural language processing technology, until all items in the first target list are matched.

[0031] Optionally, the process of comparing and matching the target matching data with the content text of all items in the second target list in sequence based on preset natural language processing technology to obtain the corresponding matching results includes:

[0032] The target matching data is processed using a preset language model to convert the target matching data into a first target vector;

[0033] The target content text of the target item element to be compared and matched in the second target list is determined, and the target content text is processed using the preset language model to convert the target content text into a second target vector.

[0034] Determine the cosine value of the angle between the first target vector and the second target vector, so as to determine the similarity between the target content text and the target matching data based on the cosine value;

[0035] If the similarity is higher than a preset similarity threshold, a matching result is obtained that indicates that there is content text in the second target list that matches the target matching data.

[0036] Optionally, after determining whether the coverage of the software test cases meets the preset coverage conditions based on the obtained matching results, the method further includes:

[0037] If it is determined that the coverage of the software test cases does not meet the preset coverage condition, feedback is given on the current determination result to obtain the review result for the determination result;

[0038] If the audit result indicates that the judgment result is correct, then it is determined that the software test case needs to be supplemented, and feedback is given based on the audit result to obtain the supplemented software test case;

[0039] If the audit result indicates that the judgment result is incorrect, the preset natural language processing technology will be optimized based on the audit result.

[0040] Secondly, this application provides a software testing apparatus, comprising:

[0041] The file determination module is used to determine the software test cases to be evaluated and to determine the corresponding source code files for the software test cases; the source code files are pre-developed software source code files with code comments based on a preset annotation setting format; the preset annotation setting format includes preset custom annotations;

[0042] The first information storage module is used to parse the plurality of source code files to obtain code comments of the development code in the plurality of source code files, and to parse the software test cases to obtain corresponding case-related information, and to save the case-related information to a preset case information list;

[0043] The second information storage module is used to perform quality evaluation on the code comments. If the evaluation result indicates that the code comments have passed the quality evaluation, then the code-related information of the several source code files is obtained based on the code comments, and the code-related information is saved to a preset code information list.

[0044] The software testing module is used to compare and match the information in the preset code information list and the preset case information list based on preset natural language processing technology, so as to determine whether the coverage of the software test cases meets the preset coverage conditions based on the matching results, and after determining that the coverage meets the preset coverage conditions, to perform software testing operations on the plurality of source code files based on the software test cases.

[0045] Each of the aforementioned related information includes corresponding functional module information and logical detail information; the preset case information list includes the first functional module list and the first logical detail list of the software test cases; the preset code information list includes the second functional module list and the second logical detail list of the plurality of source code files.

[0046] Thirdly, this application provides an electronic device, comprising:

[0047] Memory, used to store computer programs;

[0048] A processor for executing the computer program to implement the aforementioned software testing method.

[0049] Fourthly, this application provides a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned software testing method.

[0050] In this application, a software test case to be evaluated is determined, and several source code files corresponding to the software test case are identified. The source code files are pre-developed software source code files with code annotations based on a preset annotation setting format. The preset annotation setting format includes preset custom annotations. The several source code files are parsed to obtain code comments for the developed code within the several source code files. The software test case is also parsed to obtain corresponding case-related information, and this case-related information is saved to a preset case information list. A quality assessment is performed on the code comments. If the assessment result indicates that the code comments have passed the quality assessment, then the code-related information of the several source code files is obtained based on the code comments. The code-related information is saved to a preset code information list. Based on preset natural language processing technology, the information in the preset code information list and the preset test case information list is compared and matched to determine whether the coverage of the software test case meets the preset coverage condition. After determining that the coverage meets the preset coverage condition, software testing is performed on the plurality of source code files based on the software test case. The relevant information includes functional module information and logical detail information. The preset test case information list includes a first functional module list and a first logical detail list of the software test case. The preset code information list includes a second functional module list and a second logical detail list of the plurality of source code files. As can be seen from the above, this application first determines the software test cases to be evaluated and the corresponding source code files. Then, it parses the source code files to obtain code comments. The software test cases are then parsed to obtain relevant information and saved to a preset case information list. The code comments are then evaluated for quality. After passing the quality evaluation, relevant code information is extracted from the code comments and saved to the preset code information list. Finally, preset natural language processing technology is used to compare and match the information in the two lists to determine whether the coverage of the software test cases meets preset coverage conditions. If the conditions are met, software testing is performed on the source code files based on the test cases. In this way, through the above process of this application, the reliability of code information extraction is ensured by the standardized annotation format; code quality verification is performed before coverage evaluation to avoid errors in coverage evaluation due to quality issues with code comments; and the combination of natural language processing technology to achieve accurate matching between software test cases and relevant code information effectively determines the coverage of functional modules and logical details by the software test cases, ensuring case quality, improving software testing efficiency, reducing the risk of missing key functions or logic, and thus optimizing the software test case coverage evaluation process to improve the efficiency of software testing. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0052] Figure 1 This is a flowchart of a software testing method disclosed in this application;

[0053] Figure 2 This is a flowchart illustrating the timing of a software testing method disclosed in this application.

[0054] Figure 3 This is a schematic diagram of the structure of a software testing device disclosed in this application;

[0055] Figure 4 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation

[0056] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0057] Ensuring test case coverage in traditional methods relies on manual software testing followed by coverage evaluation. One approach involves analyzing the number of bugs (vulnerabilities) to assess development quality and determine whether testing needs to be strengthened, which can lead to repeated testing and extend the project timeline. Another approach involves judging the adequacy of testing based on bug escape rates, which can increase production failure rates and lead to irreversible production accidents, resulting in low efficiency in software testing.

[0058] To overcome the aforementioned technical problems, this application provides a software testing method to optimize the coverage evaluation process of software test cases and improve the efficiency of software testing.

[0059] See Figure 1 As shown in the figure, an embodiment of the present invention discloses a software testing method, including:

[0060] Step S11: Determine the software test case to be evaluated and determine the corresponding source code files for the software test case; the source code files are pre-developed software source code files with code annotations based on a preset annotation setting format; the preset annotation setting format includes preset custom annotations.

[0061] In this embodiment, the software test case to be evaluated is determined, and its corresponding source code files are identified. It should be noted that these source code files are all pre-developed software source code files with code annotations based on a preset annotation setting format using Java (a programming language) annotation technology. The preset annotation setting format includes preset custom annotations.

[0062] It should be further noted that the aforementioned preset custom annotation is specifically added to each method name in the Java development code. For example, `@autoidentify`. Based on development standards, each method must have a corresponding Chinese comment to briefly describe its implemented function and logic. The logic of the `@autoidentify` annotation is that if the method name references this annotation, the Chinese comment on the method name can be automatically read. This requires establishing a standard for developers, namely, that the function description and implementation logic must be clearly written in the Chinese comment. A corresponding fixed template can be as follows:

[0063] [Function Description]: XXXX Main Switch Configuration

[0064] [Logic 1]: XXX

[0065] [Logic 2]: XXX;

[0066] In other words, after reading the Chinese comments, the functional description and implementation logic can be distinguished by the content within the 【】 symbols in the Chinese comments.

[0067] Specifically, the software test case to be evaluated is determined, along with its corresponding target code requirement number. Based on the target code requirement number, several source code files corresponding to the software test case are identified. These several source code files are those whose filenames contain code requirement numbers that match the target code requirement number. That is, since each source code file is a class file, and a software test case may correspond to multiple source code files, this embodiment can add Chinese descriptions to the class files, specifying their corresponding code requirement codes. After determining the software test case to be evaluated, its corresponding target code requirement number can be determined, and the source code files whose filenames contain code requirement numbers that match the target code requirement number are identified; these are the several source code files corresponding to the software test case. In this way, this embodiment associates software test cases and source code files through code requirement numbers, achieving precise matching between the two. This avoids potential omissions or errors that may occur during manual searching, ensuring that test cases accurately correspond to the relevant source code files. This lays the foundation for subsequent targeted software testing based on test cases. By standardizing code comments through preset annotation formats, a unified basis is provided for obtaining code-related information through code comments, ensuring the accuracy and consistency of information extraction and improving the efficiency of software test case coverage evaluation during the software testing process.

[0068] Step S12: Parse the plurality of source code files respectively to obtain code comments of the development code in the plurality of source code files, and parse the software test cases to obtain corresponding case-related information, and save the case-related information to a preset case information list; each of the related information includes corresponding functional module information and logical detail information; the preset case information list includes the first functional module list and the first logical detail list of the software test cases.

[0069] In this embodiment, after determining the plurality of source code files and the software test cases, they are parsed respectively to obtain the code comments of the development code in the plurality of source code files and the case-related information of the software test cases, and the case-related information is saved to a preset case information list. The software test cases are typically written using Excel (spreadsheet), and include functional modules, test case numbers, test scenarios, preconditions, and expected results. Table 1 below shows an example of a software test case provided in this application.

[0070] Table 1 Software Test Case Information Table

[0071]

[0072] It should be noted that the preset case information list includes a first functional module list and a first logical detail list of the software test cases; each of the related information includes corresponding functional module information and logical detail information. For example... Figure 2 The diagram shows the flow sequence of a software testing method provided in this application. Specifically, the pre-defined custom annotations are identified in the development code of the several source code files to read the code comments of the development code based on the identification results; the functional module field of each record in the body of the software test case is read to obtain the first functional module information; the test scenario field of each record in the body is read, and semantic analysis is performed on the test scenario field based on a pre-defined language model to obtain the first logical detail information based on the analysis results; the case-related information includes the first functional module information and the first logical detail information; the first functional module information is saved to the first functional module list, and the first logical detail information is saved to the first logical detail list. In other words, the pre-defined custom annotations are first identified in the development code of the aforementioned source code files. Based on the identification results, code comments on the method names of the development code are read. Then, the software test cases are read using Java's Excel processing method, converted into a data stream, and then the functional module field of each record in the table body is read sequentially to obtain the first functional module information. Simultaneously, the test scenario field of each record is read, and semantic analysis of the test scenario field is performed using a pre-defined language model. Based on the analysis results, the actual test scenario, i.e., the first logical detail information, is obtained. Finally, the first functional module information is stored in the first functional module list, and the first logical detail information is stored in the first logical detail list. It should be noted that the case-related information includes the first functional module information and the first logical detail information. In this way, this embodiment accurately extracts code comments by identifying custom annotations, uses a language model to perform semantic analysis of test scenarios to obtain logical details, ensuring the accuracy of code comment extraction; and stores information by category in lists, providing a data foundation for subsequent comparison and matching of test cases and code, which helps improve the accuracy of matching.

[0073] Step S13: Perform a quality assessment on the code comments. If the assessment result indicates that the code comments have passed the quality assessment, then obtain the code-related information of the plurality of source code files based on the code comments, and save the code-related information to a preset code information list. The preset code information list includes a second functional module list and a second logical detail list of the plurality of source code files.

[0074] In this embodiment, in order to ensure the quality of the obtained code comments, the code comments are subjected to quality assessment and verification to obtain the corresponding assessment results. When the assessment results indicate that the code comments have passed the quality assessment, the code-related information of the several source code files is obtained according to the code comments and saved to a preset code information list.

[0075] It should be noted that the preset code information list includes a second functional module list and a second logical detail list of the plurality of source code files. Therefore, the process of obtaining code-related information from the plurality of source code files and saving it to the preset code information list is as follows: The second functional module information and the second logical detail information in the code comments are identified according to the preset annotation setting format; the code-related information includes the second functional module information and the second logical detail information; the second functional module information is saved to the second functional module list, and the second logical detail information is saved to the second logical detail list. That is, the corresponding second functional module information and second logical detail information in the code comments are identified through the preset annotation setting format, and they are saved to the second functional module list and the second logical detail list respectively. The code-related information includes the second functional module information and the second logical detail information.

[0076] It should be further pointed out that the processing flow for quality assessment of the code comments is as follows: Determine whether the code comments contain target descriptive keywords; the target descriptive keywords include a first descriptive keyword representing functional description and a second descriptive keyword representing logic; if the code comments contain the target descriptive keywords, count the number of times the second descriptive keyword appears in the code comments, and determine the total number of lines of code in the source code files; determine a target occurrence frequency threshold based on the total number of lines of code, and compare the occurrence frequency with the target occurrence frequency threshold to obtain a corresponding comparison result; if the comparison result indicates that the occurrence frequency is not lower than the target occurrence frequency threshold, then an assessment result indicating that the code comments have passed the quality assessment is obtained. That is, after obtaining the code comments, accurately check whether the code comments contain a first descriptive keyword representing functional description, such as the descriptive word "functional description," and a second descriptive keyword representing logic, such as the descriptive word "logic." If the check result indicates that they do not exist, the developers need to be prompted to write comments according to the specifications to obtain source code files with standardized comments; if the check result indicates that they exist, further examine the occurrence frequency of the second descriptive keyword to reasonably determine whether the code comments are comprehensive. Specifically, we first count the number of times (M) a logic element appears in the code comments, and then determine the total number of lines of code (P) in the source code files. Based on the principle that more logic elements require more code, we use a large amount of data for calculation to determine the average pattern: one logic element corresponds to an average of N lines of code, thus enabling reverse detection. That is, when the total number of lines of code is P, the theoretical number of times the corresponding logic element appears should be no less than the target occurrence threshold (X). The target occurrence threshold is determined based on the total number of lines of code (P), and its formula can be expressed as follows:

[0077] ;

[0078] Where X is the target occurrence frequency threshold; P is the total number of lines of code; and N is the calculated average number of lines of code corresponding to one logic. After determining the target occurrence frequency threshold, the occurrence frequency is compared with it. If it is lower than the target occurrence frequency threshold, the code comment is determined to be incomplete, and the developer needs to be prompted to verify it. If it is confirmed, the code comment is supplemented. If it is not lower than the target occurrence frequency threshold, an evaluation result indicating that the code comment has passed the quality assessment is obtained. In this way, this embodiment performs quality assessment on the code comments after obtaining them, which can avoid incomplete parsing due to non-standard code comments on code blocks, and thus incorrectly compare and obtain verification results that deviate from the actual results, thereby improving the accuracy of coverage assessment.

[0079] Step S14: Based on preset natural language processing technology, compare and match the information in the preset code information list and the preset case information list to determine whether the coverage of the software test case meets the preset coverage condition. After determining that the coverage meets the preset coverage condition, perform software testing operations on the several source code files based on the software test case.

[0080] In this embodiment, the information in the preset code information list and the preset case information list is compared and matched based on the preset natural language processing technology to obtain the corresponding matching results. Based on the matching results, it is determined whether the coverage of the software test cases meets the preset coverage conditions. If it does, software testing operations are performed on the several source code files based on the software test cases.

[0081] It should be noted that the process for comparing and matching information in the lists is as follows: The number of first elements stored in the first target list and the number of second elements stored in the corresponding second target list are determined. Wherein, if the first target list is the first functional module list, then the corresponding second target list is the second functional module list; if the first target list is the first logical detail list, then the corresponding second target list is the second logical detail list; if the number of first elements is the same as the number of second elements, then the content text of the first item in the first target list is determined as the target matching data, and based on preset natural language processing technology, the target matching data is sequentially compared and matched with the content text of all items in the second target list to obtain the corresponding matching result; if the matching result indicates that there is content text in the second target list that matches the target matching data, then the content text of the next item in the first target list is determined as the latest target matching data, and the process jumps to the step of sequentially comparing and matching the target matching data with the content text of all items in the second target list based on preset natural language processing technology, until all items in the first target list are matched. That is, first determine the number of first elements in the first target list (which can be a first functional module list or a first logical detail list) and the number of second elements in the corresponding second target list (which corresponds to a second functional module list or a second logical detail list). When the two numbers are the same, the content text of the first element in the first target list is used as the target matching data. A preset natural language processing technique is used to compare and match this content text with the content text of all elements in the second target list. If the matching result indicates that there is matching content, then the content text of the next element in the first target list is used as the new target matching data. The above matching steps are repeated until all elements in the first target list have been matched. It should be noted that during the matching process, if the matching result indicates that there is no matching content, i.e., the matching is not up to standard, it means that the software test cases are not complete enough and the coverage is insufficient, requiring further supplementation. Specifically, the functional module list can be compared and matched first. After it passes the test benchmark check, that is, after the matching results of the above steps all have matching content, the logical detail list can be compared and matched. If the matching degree does not reach 100%, it means that the test case scenario coverage is not up to standard, and the judgment result is that the coverage of the software test case does not meet the preset coverage condition.

[0082] It should be further noted that the process of comparing and matching the target matching data with the content text of all items in the second target list in sequence based on preset natural language processing technology is as follows: The target matching data is processed using a preset language model to convert it into a first target vector; the target content text of the target item element to be compared and matched in the second target list is determined, and the target content text is processed using the preset language model to convert it into a second target vector; the cosine value of the angle between the first target vector and the second target vector is determined, and the similarity between the target content text and the target matching data is determined based on the cosine value; if the similarity is higher than a preset similarity threshold, a matching result is obtained indicating that there is content text in the second target list that matches the target matching data. The preset language model can be a BERT model (a pre-trained language model). That is, the target matching data is processed by Chinese word segmentation and sub-word segmentation using a preset language model, and converted into a first target vector with a default dimension, such as a 768-dimensional vector. At the same time, the target content text of the target item element to be matched in the second target list is converted into a second target vector in the same way. For example, "deferred principal and interest repayment" is converted into [0.12, -0.45, 0.78, ..., 0.32] (768-dimensional), and "deferred principal and interest repayment" is converted into [0.15, -0.41, 0.72, ..., 0.30]. Then, the cosine value of the angle between the two vectors is calculated to determine their similarity. Specifically, the direction of the two texts in the semantic space is compared. The formula for calculating the cosine value can be as follows:

[0083] ;

[0084] in, That is, the cosine value; A and B represent the first target vector and the second target vector, respectively; The similarity is then subjected to norm operations. After determining the similarity, if the similarity is higher than a preset similarity threshold, it is determined that there is content text in the second target list that matches the target matching data.

[0085] It should be noted that the processing flow after determining whether the preset coverage condition is met based on the matching result is as follows: If it is determined that the coverage of the software test case does not meet the preset coverage condition, feedback is given on the current determination result to obtain an audit result for the determination result; if the audit result indicates that the determination result is correct, it is determined that the software test case needs to be supplemented, and feedback is given based on the audit result to obtain the supplemented software test case; if the audit result indicates that the determination result is incorrect, the preset natural language processing technology is optimized based on the audit result. In other words, if the matching results indicate that the coverage of the software test cases is insufficient, feedback can be provided on the current judgment result. Specifically, a problem list can be generated, prompting testers to intervene manually. The comprehensiveness of the test cases can be manually reviewed. If it is consistent with the facts, the conclusion of this problem is confirmed as reviewed, and the relevant responsible person is prompted to continue supplementing test cases. At the same time, a qualified review result is performed on the judgment result. If the review result is qualified, that is, the judgment result is correct, then it is determined that the software test cases need to be supplemented, and feedback is provided based on the review result to obtain the supplemented software test cases. If it is inconsistent with the facts, the conclusion of the problem is confirmed as "inconsistent with the facts, no action required," and a review result indicating that further improvement is needed is performed on the automatic monitoring model. If the review result indicates that further improvement is needed, that is, the judgment result is incorrect, then manual intervention is required to optimize the preset natural language processing technology based on the review result. In this way, during the comparison and matching process, this embodiment first confirms whether the number of elements is consistent. By judging consistency, the basic conditions for matching are guaranteed, thus improving the efficiency of comparison and matching. Natural language processing technology is used to achieve accurate comparison of text content. The sequential item-by-item matching method ensures the comprehensiveness of the matching, which helps to improve the accuracy of the matching results and thus ensures the effectiveness of software testing. By using a language model to convert text into vectors and then calculating similarity using cosine values, the semantic relationships between texts can be captured more accurately, avoiding the deviations that may occur based solely on literal matching, and improving the accuracy and flexibility of text matching. The introduction of a review process allows for reasonable identification of cases with insufficient coverage, ensuring the necessity and relevance of test case supplementation, avoiding invalid work caused by misjudgment, and continuously improving the matching mechanism between test cases and code information. This ensures that software testing can more comprehensively and accurately cover relevant functions and logic, ultimately improving the quality and efficiency of software testing.

[0086] As can be seen from the above, the embodiments of this application first determine the software test cases to be evaluated and the corresponding source code files, then parse the source code files to obtain code comments of the development code, parse the software test cases to obtain case-related information and save it to a preset case information list, perform quality evaluation on the code comments, extract code-related information from the code comments after passing the quality evaluation, save it to a preset code information list, and then use preset natural language processing technology to compare and match the information in the two lists to determine whether the coverage of the software test cases meets the preset coverage conditions. After determining that it meets the conditions, software testing is performed on the source code files based on the test cases. In this way, through the above-described process of the embodiments of this application, on the one hand, the association between software test cases and source code files is achieved through code requirement numbers, realizing accurate matching between the two, avoiding possible omissions or errors due to manual searching, and ensuring that test cases can accurately correspond to relevant source code files, laying the foundation for subsequent targeted software testing based on test cases; on the other hand, by standardizing code comments through preset annotation formats, a unified basis is provided for obtaining code-related information through code comments, ensuring the accuracy and consistency of information extraction, and improving the efficiency of software test case coverage evaluation during the software testing process; on the other hand, by accurately extracting code comments through the identification of custom annotations, and using a large language model to perform semantic analysis of test scenarios to obtain logical details, the accuracy of code comment extraction is guaranteed; on the other hand, by storing information through list classification, a data foundation is provided for subsequent comparison and matching of test cases and code, which helps to improve the accuracy of matching; and on the other hand, after obtaining code comments, quality evaluation is performed to avoid incomplete parsing due to non-standard code comments on code blocks, resulting in incorrect comparisons that deviate from the actual results. To improve the accuracy of coverage assessment, the following measures are taken: First, during the comparison and matching process, the consistency of element count is confirmed first. This consistency check ensures the basic conditions for matching and improves the efficiency of comparison and matching. Second, natural language processing technology is used to achieve accurate comparison of text content. The sequential item-by-item matching method ensures the comprehensiveness of the matching, which helps improve the accuracy of the matching results and thus ensures the effectiveness of software testing. Third, by using a language model to convert text into vectors and then calculating similarity using cosine values, the semantic relationships between texts can be captured more accurately, avoiding the deviations that may occur based solely on literal matching and improving the accuracy and flexibility of text matching. Fourth, an auditing process is introduced to reasonably identify cases where coverage does not meet the standards, ensuring the necessity and relevance of test case supplementation, avoiding invalid work caused by misjudgment, and continuously improving the matching mechanism between test cases and code information. This ensures that software testing can more comprehensively and accurately cover relevant functions and logic, ultimately improving the quality and efficiency of software testing, and thus optimizing the software test case coverage assessment process to improve the efficiency of software testing work.

[0087] Accordingly, see Figure 3 As shown in the illustration, this application also provides a software testing apparatus, including:

[0088] The file determination module 11 is used to determine the software test case to be evaluated and to determine several source code files corresponding to the software test case; the source code files are pre-developed software source code files with code comments based on a preset annotation setting format; the preset annotation setting format includes preset custom annotations;

[0089] The first information storage module 12 is used to parse the plurality of source code files respectively to obtain code comments of the development code in the plurality of source code files, and to parse the software test cases to obtain corresponding case-related information, and to save the case-related information to a preset case information list.

[0090] The second information storage module 13 is used to perform quality evaluation on the code comments. If the evaluation result indicates that the code comments have passed the quality evaluation, then the code-related information of the several source code files is obtained based on the code comments, and the code-related information is saved to a preset code information list.

[0091] Software testing module 14 is used to compare and match the information in the preset code information list and the preset case information list based on preset natural language processing technology, so as to determine whether the coverage of the software test cases meets the preset coverage conditions based on the obtained matching results, and after determining that the coverage meets the preset coverage conditions, to perform software testing operations on the plurality of source code files based on the software test cases.

[0092] Each of the aforementioned related information includes corresponding functional module information and logical detail information; the preset case information list includes the first functional module list and the first logical detail list of the software test cases; the preset code information list includes the second functional module list and the second logical detail list of the plurality of source code files.

[0093] As can be seen from the above, the embodiments of this application first determine the software test cases to be evaluated and the corresponding source code files, then parse the source code files to obtain code comments of the development code, parse the software test cases to obtain case-related information and save it to a preset case information list, perform quality evaluation on the code comments, extract code-related information from the code comments after passing the quality evaluation, save it to a preset code information list, and then use preset natural language processing technology to compare and match the information in the two lists to determine whether the coverage of the software test cases meets the preset coverage conditions. After determining that it meets the conditions, software testing is performed on the source code files based on the test cases. In this way, through the above-described process of the embodiments of this application, the reliability of code information extraction is ensured by the standardized annotation format; code quality verification is performed before coverage evaluation to avoid errors in coverage evaluation due to quality issues with code comments; and the combination of natural language processing technology to achieve accurate matching between software test cases and code-related information can effectively determine the coverage of functional modules and logical details by software test cases, ensure test case quality, improve software testing efficiency, reduce the risk of missing key functions or logic, and thus optimize the coverage evaluation process of software test cases to improve the efficiency of software testing.

[0094] In some specific embodiments, the file determination module 11 may specifically include:

[0095] The numbering determination unit is used to determine the software test case to be evaluated and the corresponding target code requirement number.

[0096] The file determination unit is used to determine a plurality of source code files corresponding to the software test case based on the target code requirement number; the plurality of source code files are source code files whose file names contain code requirement numbers that are consistent with the target code requirement number.

[0097] In some specific embodiments, the first information storage module 12 may specifically include:

[0098] An annotation recognition unit is used to recognize the preset custom annotations in the development code of the plurality of source code files respectively, so as to read the code comments of the development code according to the recognition results.

[0099] The field reading unit is used to read the functional module field of each record in the body of the software test case to obtain the first functional module information;

[0100] The field analysis unit is used to read the test scenario fields of each record in the table body, and perform semantic analysis on the test scenario fields based on a preset language model, so as to obtain the first logical detail information based on the analysis results; the case-related information includes the first functional module information and the first logical detail information.

[0101] The first information storage unit is used to save the first functional module information to the first functional module list and the first logical detail information to the first logical detail list.

[0102] Accordingly, the second information storage module 13 may specifically include:

[0103] An information recognition unit is used to identify the second functional module information and the second logical detail information in the code comment according to the preset annotation setting format; the code-related information includes the second functional module information and the second logical detail information.

[0104] The second information storage unit is used to save the second functional module information to the second functional module list and the second logical detail information to the second logical detail list.

[0105] In some specific embodiments, the second information storage module 13 may specifically include:

[0106] A condition judgment unit is used to determine whether there are target description keywords in the code comments; the target description keywords include a first description keyword representing the functional description and a second description keyword representing the logic.

[0107] The line count determination unit is used to count the number of times the second descriptive keyword appears in the code comment if the target descriptive keyword exists in the code comment, and to determine the total number of lines of code in the plurality of source code files;

[0108] The frequency comparison unit is used to determine the target occurrence frequency threshold based on the total number of lines of code, and compare the occurrence frequency with the target occurrence frequency threshold to obtain the corresponding comparison result;

[0109] The evaluation result determination unit is used to obtain an evaluation result indicating that the code annotation has passed the quality evaluation if the comparison result indicates that the occurrence frequency is not lower than the target occurrence frequency threshold.

[0110] In some specific embodiments, the software testing module 14 may specifically include:

[0111] The element count determination unit is used to determine the number of first elements stored in the first target list and the number of second elements stored in the corresponding second target list; wherein, if the first target list is the first functional module list, then the corresponding second target list is the second functional module list; if the first target list is the first logical detail list, then the corresponding second target list is the second logical detail list.

[0112] The data matching submodule is used to determine the content text of the first item in the first target list as the target matching data if the number of the first element is the same as the number of the second element, and to compare and match the target matching data with the content text of all items in the second target list in turn based on the preset natural language processing technology to obtain the corresponding matching results.

[0113] The step jump unit is used to determine the content text of the next item in the first target list as the latest target matching data if the matching result indicates that there is content text in the second target list that matches the target matching data, and jump to the step of comparing and matching the target matching data with the content text of all items in the second target list in turn based on the preset natural language processing technology, until all items in the first target list are matched.

[0114] In some specific implementations, the data matching submodule may specifically include:

[0115] A data processing unit is used to process the target matching data using a preset language model to convert the target matching data into a first target vector;

[0116] The text processing unit is used to determine the target content text of the target item element to be compared and matched in the second target list, and to process the target content text using the preset language model to convert the target content text into a second target vector.

[0117] The cosine value determination unit is used to determine the cosine value of the angle between the first target vector and the second target vector, so as to determine the similarity between the target content text and the target matching data based on the cosine value;

[0118] The matching result determination unit is used to obtain a matching result indicating that there is content text in the second target list that matches the target matching data if the similarity is higher than a preset similarity threshold.

[0119] In some specific embodiments, the software testing apparatus may further include:

[0120] The judgment result feedback unit is used to provide feedback on the current judgment result if the coverage of the software test case does not meet the preset coverage condition, so as to obtain the review result for the judgment result.

[0121] The audit result feedback unit is used to determine that the software test case needs to be supplemented if the audit result indicates that the judgment result is correct, and to provide feedback based on the audit result to obtain the supplemented software test case.

[0122] The technology optimization unit is used to optimize the preset natural language processing technology based on the audit result if the audit result indicates that the judgment result is incorrect.

[0123] Furthermore, embodiments of this application also disclose an electronic device, Figure 4 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the software testing method disclosed in any of the foregoing embodiments. Furthermore, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0124] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0125] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0126] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including computer programs capable of performing the software testing methods executed by the electronic device 20 as disclosed in any of the foregoing embodiments, the computer program 222 may further include computer programs capable of performing other specific tasks.

[0127] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned disclosed software testing method. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0128] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0129] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0130] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0131] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0132] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A software testing method, characterized in that, include: Identify the software test cases to be evaluated and determine the corresponding source code files for each software test case; The source code file is a pre-developed software source code file with code annotations based on a preset annotation setting format; The preset annotation setting format includes preset custom annotations; The source code files are parsed to obtain code comments for the development code in the source code files, and the software test cases are parsed to obtain corresponding case-related information, and the case-related information is saved to a preset case information list; A quality assessment is performed on the code comments. If the assessment result indicates that the code comments have passed the quality assessment, then the code-related information of the several source code files is obtained based on the code comments, and the code-related information is saved to a preset code information list. Based on the preset natural language processing technology, the information in the preset code information list and the preset case information list is compared and matched to determine whether the coverage of the software test case meets the preset coverage condition. After determining that the coverage meets the preset coverage condition, software testing operations are performed on the several source code files based on the software test case. Each of the aforementioned related information includes corresponding functional module information and logical detail information; the preset case information list includes the first functional module list and the first logical detail list of the software test cases; the preset code information list includes the second functional module list and the second logical detail list of the plurality of source code files.

2. The software testing method according to claim 1, characterized in that, The process of determining the current software test case to be evaluated and determining the corresponding source code files for the software test case includes: Identify the software test cases to be evaluated and their corresponding target code requirement numbers; Based on the target code requirement number, several source code files corresponding to the software test case are determined; the several source code files are source code files whose file names contain code requirement numbers that match the target code requirement number.

3. The software testing method according to claim 1, characterized in that, The process of parsing the plurality of source code files to obtain code comments for the development code in the plurality of source code files, parsing the software test cases to obtain corresponding test case-related information, and saving the test case-related information to a preset test case information list includes: The preset custom annotations are identified in the development code of the aforementioned source code files respectively, so as to read the code comments of the development code based on the obtained identification results; Read the functional module field of each record in the body of the software test case to obtain the first functional module information; The test scenario fields of each record in the table body are read, and semantic analysis is performed on the test scenario fields based on a preset language model to obtain the first logical detail information based on the analysis results; the case-related information includes the first functional module information and the first logical detail information. The first functional module information is saved to the first functional module list, and the first logical detail information is saved to the first logical detail list; Accordingly, the step of obtaining code-related information of the plurality of source code files based on the code comments and saving the code-related information to a preset code information list includes: The second functional module information and the second logical detail information in the code comments are identified according to the preset annotation setting format; the code-related information includes the second functional module information and the second logical detail information; The second functional module information is saved to the second functional module list, and the second logical detail information is saved to the second logical detail list.

4. The software testing method according to claim 1, characterized in that, The quality assessment of the code comments includes: Determine whether target description keywords exist in the code comments; the target description keywords include first description keywords representing functional descriptions and second description keywords representing logical descriptions; If the target description keyword exists in the code comments, the number of times the second description keyword appears in the code comments is counted, and the total number of lines of code in the plurality of source code files is determined; A target occurrence frequency threshold is determined based on the total number of lines of code, and the occurrence frequency is compared with the target occurrence frequency threshold to obtain the corresponding comparison result; If the comparison result indicates that the occurrence frequency is not lower than the target occurrence frequency threshold, then an evaluation result indicating that the code annotation has passed the quality assessment is obtained.

5. The software testing method according to claim 1, characterized in that, The step of comparing and matching information in the preset code information list and the preset case information list based on preset natural language processing technology includes: Determine the number of first elements stored in the first target list and the number of second elements stored in the corresponding second target list; wherein, if the first target list is the first functional module list, then the corresponding second target list is the second functional module list; if the first target list is the first logical detail list, then the corresponding second target list is the second logical detail list. If the number of the first element is the same as the number of the second element, the content text of the first item in the first target list is determined as the target matching data, and based on the preset natural language processing technology, the target matching data is compared and matched with the content text of all items in the second target list in turn to obtain the corresponding matching result; If the matching result indicates that there is content text in the second target list that matches the target matching data, then the content text of the next item in the first target list is determined as the latest target matching data, and the process jumps to the step of comparing and matching the target matching data with the content text of all items in the second target list in turn based on preset natural language processing technology, until all items in the first target list are matched.

6. The software testing method according to claim 5, characterized in that, The process of comparing and matching the target matching data with the content text of all items in the second target list in turn, based on preset natural language processing technology, to obtain the corresponding matching results, includes: The target matching data is processed using a preset language model to convert the target matching data into a first target vector; The target content text of the target item element to be compared and matched in the second target list is determined, and the target content text is processed using the preset language model to convert the target content text into a second target vector. Determine the cosine value of the angle between the first target vector and the second target vector, so as to determine the similarity between the target content text and the target matching data based on the cosine value; If the similarity is higher than a preset similarity threshold, a matching result is obtained that indicates that there is content text in the second target list that matches the target matching data.

7. The software testing method according to any one of claims 1 to 6, characterized in that, After determining whether the coverage of the software test cases meets the preset coverage conditions based on the obtained matching results, the method further includes: If it is determined that the coverage of the software test cases does not meet the preset coverage condition, feedback is given on the current determination result to obtain the review result for the determination result; If the audit result indicates that the judgment result is correct, then it is determined that the software test case needs to be supplemented, and feedback is given based on the audit result to obtain the supplemented software test case; If the audit result indicates that the judgment result is incorrect, the preset natural language processing technology will be optimized based on the audit result.

8. A software testing apparatus, characterized in that, include: The file determination module is used to determine the software test cases to be evaluated and to determine the source code files corresponding to the software test cases. The source code file is a pre-developed software source code file with code annotations based on a preset annotation setting format; The preset annotation setting format includes preset custom annotations; The first information storage module is used to parse the plurality of source code files to obtain code comments of the development code in the plurality of source code files, and to parse the software test cases to obtain corresponding case-related information, and to save the case-related information to a preset case information list; The second information storage module is used to perform quality evaluation on the code comments. If the evaluation result indicates that the code comments have passed the quality evaluation, then the code-related information of the several source code files is obtained based on the code comments, and the code-related information is saved to a preset code information list. The software testing module is used to compare and match the information in the preset code information list and the preset case information list based on preset natural language processing technology, so as to determine whether the coverage of the software test cases meets the preset coverage conditions based on the matching results, and after determining that the coverage meets the preset coverage conditions, to perform software testing operations on the plurality of source code files based on the software test cases. Each of the aforementioned related information includes corresponding functional module information and logical detail information; the preset case information list includes the first functional module list and the first logical detail list of the software test cases; the preset code information list includes the second functional module list and the second logical detail list of the plurality of source code files.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the software testing method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, Used to store computer programs; wherein, when the computer programs are executed by a processor, they implement the software testing method as described in any one of claims 1 to 7.

Images