Automatic testing method based on natural language processing

Abstract

The invention discloses an automatic test method based on natural language processing, which comprises the following steps: S1, generating a test case: performing semantic analysis on a software requirement document or a test requirement by utilizing a natural language processing technology, extracting key information, and combining the extracted test information and a preset test case template to generate a test case; a preliminary test case is automatically generated; the key information comprises a test purpose, an operation step and an expected result; s2, analyzing the test case; s3, generating a test script; s4, executing the test script; and S5, analyzing a test result. According to the automatic test method based on natural language processing, the test requirements described by the natural language can be automatically analyzed, the test case is generated, the test is automatically executed, and the test report is finally generated, so that the software test efficiency is improved, the workload of manually compiling and executing the test case is reduced, and the test efficiency is improved. And the coverage rate and the accuracy of the test are improved, more software defects can be found, and the quality of software products is guaranteed.

Description

Technical Field

[0001] The present invention belongs to the technical field of software testing, and in particular relates to an automatic testing method based on natural language processing. Background Art

[0002] After decades of development in the field of software automation testing, many excellent automation testing frameworks and tools have emerged at home and abroad. The most representative ones are as follows:

[0003] 1. Selenium, an automated testing tool for human-computer interaction interfaces;

[0004] 2. Abandon the interactive interface and directly use Postman, a tool that captures POST / GET interfaces and other custom interfaces, to simulate browser requests for automated testing;

[0005] 3. Robotframework, a keyword-driven framework that integrates automated testing of human-computer interaction interfaces and automated testing of simulated browser requests;

[0006] 4. TestNG, a testing framework used to integrate and manage the execution of automated test cases.

[0007] The above four excellent test frameworks and tools basically cover and represent the design philosophy and ideas of automated test framework tools in the testing industry, providing long-term and outstanding support for ensuring the quality of software engineering, improving test efficiency, and reducing software engineering costs.

[0008] In the traditional software testing process, the writing and execution of test cases often require a lot of manual operations, which is not only inefficient but also easily affected by personal subjective factors, resulting in unstable test results. With the rapid development of natural language processing technology, it has become a trend to apply it to the field of software testing to realize the automatic generation and execution of test cases. However, the existing testing methods based on natural language processing still have certain limitations, such as low parsing accuracy and incomplete test coverage. For this reason, we propose an automated testing method based on natural language processing. Summary of the invention

[0009] The purpose of the present invention is to provide an automated testing method based on natural language processing to solve the problems raised in the above background technology.

[0010] To achieve the above object, the present invention provides the following technical solution: an automated testing method based on natural language processing, comprising the following steps:

[0011] S1. Generate test cases: Use natural language processing technology to perform semantic analysis on software requirement documents or test requirements, extract key information, and automatically generate preliminary test cases by combining the extracted test information with a preset test case template; the key information includes test purpose, operation steps, and expected results;

[0012] S2. Parsing test cases: Use natural language processing algorithms to parse the generated test cases, extract key information such as test purpose, operation steps, and expected results, and convert them into structured data;

[0013] S3. Generate test script: Automatically generate executable test script based on the structured data obtained by parsing; the test script includes configuration of the test environment, preparation of test data, execution of test steps and verification of test results;

[0014] S4. Execute the test script: execute the generated test script in the test environment, perform automated testing on the software system, and record the test log during the testing process;

[0015] S5. Analyze test results: Analyze the test results generated during the test execution process, determine whether the test passed, and generate a test report.

[0016] Preferably, after generating the test cases, S1 also includes test case optimization, and the test case optimization includes using reinforcement learning and genetic algorithm optimization techniques to intelligently optimize the generated test cases, improve the coverage and fault detection capabilities of the test cases, and identify and eliminate redundant or inefficient test cases by analyzing historical test execution results, and increase test cases for new features and potential defects.

[0017] Preferably, the S2 also includes preprocessing test cases before parsing the test cases, and the preprocessing test cases includes preprocessing the collected test cases, and the preprocessing includes removing irrelevant information, word segmentation and part-of-speech tagging, and understanding and parsing the test cases through a natural language processing algorithm.

[0018] Preferably, the specific steps of generating the test report in S5 are:

[0019] S501. Capture test result data: When the automated test script is running, use a dedicated logging function or tool to capture various data during the test process, including whether the test case passes or fails, execution time, performance indicators, and abnormal error information;

[0020] S502. Preliminary processing of test results: cleaning and formatting the captured test result data, removing useless information, ensuring data quality and consistency, and storing the processed data in a database for subsequent analysis and report generation;

[0021] S503. Analyze test results: Use data analytics and machine learning techniques to conduct in-depth analysis of test results to identify patterns, trends, and potential issues in the test;

[0022] S504. Generate test report: Based on the analysis results, use templates or automated reporting tools to generate a detailed test report; the report includes a test overview, key indicator summary, detailed test results, defect list, performance evaluation, risk assessment, test coverage information, and charts and visualization elements.

[0023] Preferably, after S504, the step further includes:

[0024] S505. Distribute test reports: Automatically send the generated test reports to relevant developers and testers via email, instant messaging, and system notifications to ensure that the information contained in the reports is accurate and clear;

[0025] S506. Feedback loop: Include feedback mechanisms in reports, such as links to defect tracking systems, questionnaires, or direct discussion areas, to enable team members to provide feedback and improvement suggestions, and use this feedback to optimize the test process and test cases.

[0026] Preferably, S5 further includes result feedback, which includes in-depth analysis of the test results and use of machine learning and data mining techniques to identify potential software defects, performance bottlenecks and security vulnerabilities.

[0027] Preferably, the step of generating the test case template in S1 includes: classifying the template type, function template and data interaction mode template in sequence, and then defining an interface for receiving word slot parameters through template code according to the above classification to receive the test object entity name and test data, and labeling the template code according to the type classification, and matching the labeled template code through natural language processing technology to obtain the test case template.

[0028] Preferably, the natural language processing technology in S1 includes entropy calculation of decision tree, and the specific formula is as follows:

[0029]

[0030] Among them, S represents the sample set of the current node; c represents the number of categories; p i It represents the probability of the i-th element appearing in the set S; Entropy(S): It represents the entropy of the set S, which is used to measure the uncertainty or confusion of the elements in the set.

[0031] Preferably, the test case optimization includes an optimization technique of a genetic algorithm, and the fitness function of the genetic algorithm includes the following formula:

[0032]

[0033] Among them, x: represents a possible solution or individual; f(x) represents the objective function value of solution x, such as test coverage or fault detection rate; Fitness(x): represents the fitness of solution x, which is used to evaluate the quality of the solution.

[0034] Preferably, the natural language processing algorithm in S2 includes an ensemble learning method of random forests, and the specific formula is:

[0035]

[0036] Where n represents the number of decision trees in the random forest; t i Represents the prediction result of the i-th tree;

[0037] Average Prediction represents the average of all tree prediction results and is used for the final prediction in ensemble learning.

[0038] Compared with the prior art, the present invention has the following beneficial effects:

[0039] (1) The automatic testing method based on natural language processing of the present invention can automatically parse the test requirements described in natural language, generate test cases, and automatically execute the test, and finally generate a test report, which not only improves the efficiency of software testing and reduces the workload of manually writing and executing test cases, but also improves the coverage and accuracy of the test, helps to discover more software defects, and ensures the quality of software products;

[0040] (2) The present invention realizes automatic parsing and conversion of test cases through natural language processing technology, thereby improving test efficiency and accuracy. The generated test scripts have high readability and maintainability, making it convenient for testers to modify and expand them. It can realize automatic analysis of test results and reduce the workload of testers. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 is a flow chart of the present invention;

[0042] Figure 2 It is a specific flow chart of S5 of the present invention. DETAILED DESCRIPTION

[0043] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0044] See also Figure 1-Figure 2 The present invention provides a technical solution: an automated testing method based on natural language processing, comprising the following steps:

[0045] S1. Generate test cases: Use natural language processing technology to perform semantic analysis on software requirement documents or test requirements, extract key information, and automatically generate preliminary test cases by combining the extracted test information with a preset test case template; the key information includes test purpose, operation steps, and expected results;

[0046] S2. Parsing test cases: Use natural language processing algorithms to parse the generated test cases, extract key information such as test purpose, operation steps, and expected results, and convert them into structured data;

[0047] S3. Generate test script: Automatically generate executable test script based on the structured data obtained by parsing; the test script includes configuration of the test environment, preparation of test data, execution of test steps and verification of test results;

[0048] S4. Execute the test script: execute the generated test script in the test environment, perform automated testing on the software system, and record the test log during the testing process;

[0049] S5. Analyze test results: Analyze the test results generated during the test execution process, determine whether the test passed, and generate a test report.

[0050] In this embodiment, preferably, after the S1 generates the test cases, it also includes test case optimization, and the test case optimization includes using reinforcement learning and genetic algorithm optimization techniques to intelligently optimize the generated test cases, improve the coverage and fault detection capabilities of the test cases, and identify and eliminate redundant or inefficient test cases by analyzing historical test execution results, and increase test cases for new features and potential defects.

[0051] In this embodiment, preferably, the S2 also includes preprocessing test cases before parsing the test cases, and the preprocessing test cases includes preprocessing the collected test cases, and the preprocessing includes removing irrelevant information, word segmentation and part-of-speech tagging, and understanding and parsing the test cases through a natural language processing algorithm.

[0052] In this embodiment, preferably, the specific steps of generating the test report in S5 are:

[0053] S501. Capture test result data: When the automated test script is running, use a dedicated logging function or tool to capture various data during the test process, including whether the test case passes or fails, execution time, performance indicators, and abnormal error information;

[0054] S502. Preliminary processing of test results: cleaning and formatting the captured test result data, removing useless information, ensuring data quality and consistency, and storing the processed data in a database for subsequent analysis and report generation;

[0055] S503. Analyze test results: Use data analytics and machine learning techniques to conduct in-depth analysis of test results to identify patterns, trends, and potential issues in the test;

[0056] S504. Generate test report: Based on the analysis results, use templates or automated reporting tools to generate a detailed test report; the report includes a test overview, key indicator summary, detailed test results, defect list, performance evaluation, risk assessment, test coverage information, and charts and visualization elements.

[0057] In this embodiment, preferably, after S504, the following steps are further performed:

[0058] S505. Distribute test reports: Automatically send the generated test reports to relevant developers and testers via email, instant messaging, and system notifications to ensure that the information contained in the reports is accurate and clear;

[0059] S506. Feedback loop: Include feedback mechanisms in reports, such as links to defect tracking systems, questionnaires, or direct discussion areas, to enable team members to provide feedback and improvement suggestions, and use this feedback to optimize the test process and test cases.

[0060] In this embodiment, preferably, S5 further includes result feedback, and the result feedback includes in-depth analysis of the test results, and using machine learning and data mining techniques to identify potential software defects, performance bottlenecks, and security vulnerabilities.

[0061] In this embodiment, preferably, the step of generating the test case template in S1 includes: classifying the template type, function template and data interaction mode template in sequence, and then defining an interface for receiving word slot parameters through template code according to the above classification to receive the test object entity name and test data, and labeling the template code according to the type classification, and matching the labeled template code through natural language processing technology to obtain the test case template.

[0062] In this embodiment, preferably, the natural language processing technology in S1 includes entropy calculation of a decision tree, and the specific formula is as follows:

[0063]

[0064] Among them, S represents the sample set of the current node; c represents the number of categories; p i It represents the probability of the i-th element appearing in the set S; Entropy(S): It represents the entropy of the set S, which is used to measure the uncertainty or confusion of the elements in the set.

[0065] In this embodiment, preferably, the test case optimization includes an optimization technique of a genetic algorithm, and the fitness function of the genetic algorithm includes the following formula:

[0066]

[0067] Among them, x: represents a possible solution or individual; f(x) represents the objective function value of solution x, such as test coverage or fault detection rate; Fitness(x): represents the fitness of solution x, which is used to evaluate the quality of the solution.

[0068] In this embodiment, preferably, the natural language processing algorithm in S2 includes an ensemble learning method of random forests, and the specific formula is:

[0069]

[0070] Where n represents the number of decision trees in the random forest; t i Represents the prediction result of the i-th tree;

[0071] Average Prediction represents the average of all tree prediction results and is used for the final prediction in ensemble learning.

[0072] Principles and advantages of the present invention:

[0073] The automatic testing method based on natural language processing of the present invention can automatically parse the test requirements described in natural language, generate test cases, and automatically execute the test, and finally generate a test report, which not only improves the efficiency of software testing, reduces the workload of manually writing and executing test cases, but also improves the coverage and accuracy of the test, helps to discover more software defects, and ensures the quality of software products;

[0074] The present invention realizes automatic parsing and conversion of test cases through natural language processing technology, improves test efficiency and accuracy, and the generated test scripts have high readability and maintainability, which is convenient for testers to modify and expand, can realize automatic analysis of test results, and reduces the workload of testers.

[0075] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An automated testing method based on natural language processing, characterized in that: The following steps are involved: S1. Generate test cases: Use natural language processing technology to perform semantic analysis on software requirement documents or test requirements, extract key information, and automatically generate preliminary test cases by combining the extracted test information with a preset test case template; the key information includes test purpose, operation steps, and expected results; S2. Parsing test cases: Use natural language processing algorithms to parse the generated test cases, extract key information such as test purpose, operation steps, and expected results, and convert them into structured data; S3. Generate test script: Automatically generate executable test script based on the structured data obtained by parsing; the test script includes configuration of the test environment, preparation of test data, execution of test steps and verification of test results; S4. Execute the test script: execute the generated test script in the test environment, perform automated testing on the software system, and record the test log during the testing process; S5. Analyze test results: Analyze the test results generated during the test execution process, determine whether the test passed, and generate a test report.

2. The automatic testing method based on natural language processing according to claim 1, characterized in that: After S1 generates test cases, it also includes test case optimization. The test case optimization includes using reinforcement learning and genetic algorithm optimization techniques to intelligently optimize the generated test cases, improve the coverage and fault detection capabilities of the test cases, and identify and eliminate redundant or inefficient test cases by analyzing historical test execution results, and increase test cases for new features and potential defects.

3. The automatic testing method based on natural language processing according to claim 1, characterized in that: The S2 test case analysis also includes preprocessing the test cases, wherein the preprocessing test cases includes preprocessing the collected test cases, wherein the preprocessing includes removing irrelevant information, word segmentation and part-of-speech tagging, and understanding and parsing the test cases through a natural language processing algorithm.

4. The automatic testing method based on natural language processing according to claim 1, characterized in that: The specific steps of generating the test report in S5 are: S501. Capture test result data: When the automated test script is running, use a dedicated logging function or tool to capture various data during the test process, including whether the test case passes or fails, execution time, performance indicators, and abnormal error information; S502. Preliminary processing of test results: cleaning and formatting the captured test result data, removing useless information, ensuring data quality and consistency, and storing the processed data in a database for subsequent analysis and report generation; S503. Analyze test results: Use data analytics and machine learning techniques to conduct in-depth analysis of test results to identify patterns, trends, and potential issues in the test; S504. Generate test report: Based on the analysis results, use templates or automated reporting tools to generate a detailed test report; the report includes a test overview, key indicator summary, detailed test results, defect list, performance evaluation, risk assessment, test coverage information, and charts and visualization elements.

5. The automatic testing method based on natural language processing according to claim 4 is characterized in that: After S504, the following steps are also included: S505. Distribute test reports: Automatically send the generated test reports to relevant developers and testers via email, instant messaging, and system notifications to ensure that the information contained in the reports is accurate and clear; S506. Feedback loop: Include feedback mechanisms in reports, such as links to defect tracking systems, questionnaires, or direct discussion areas, to enable team members to provide feedback and improvement suggestions, and use this feedback to optimize the test process and test cases.

6. The automatic testing method based on natural language processing according to claim 1, characterized in that: The S5 also includes result feedback, which includes in-depth analysis of the test results and the use of machine learning and data mining techniques to identify potential software defects, performance bottlenecks and security vulnerabilities.

7. The automatic testing method based on natural language processing according to claim 1 is characterized in that: The generation step of the test case template in S1 includes: classifying the template type, function template and data interaction mode template in sequence, and then defining an interface for receiving word slot parameters through template code according to the above classification to receive the test object entity name and test data, and labeling the template code according to the type classification. The labeled template code is matched through natural language processing technology to obtain a test case template.

8. The automatic testing method based on natural language processing according to claim 1, characterized in that: The natural language processing technology in S1 includes entropy calculation of decision tree, and the specific formula is as follows: Among them, S represents the sample set of the current node; c represents the number of categories; p i It represents the probability of the i-th element appearing in the set S; Entropy(S): It represents the entropy of the set S, which is used to measure the uncertainty or confusion of the elements in the set.

9. The automatic testing method based on natural language processing according to claim 2, characterized in that: The test case optimization includes an optimization technique of a genetic algorithm, and the fitness function of the genetic algorithm includes the following formula: Among them, x: represents a possible solution or individual; f(x) represents the objective function value of solution x, such as test coverage or fault detection rate; Fitness(x): represents the fitness of solution x, which is used to evaluate the quality of the solution.

10. The automatic testing method based on natural language processing according to claim 1, characterized in that: The natural language processing algorithm in S2 includes an integrated learning method of random forests, and the specific formula is: Where n represents the number of decision trees in the random forest; t i Represents the prediction result of the i-th tree; Average Prediction represents the average of all tree prediction results and is used for the final prediction in ensemble learning.

Images