Automatic software testing method and system based on artificial intelligence

Abstract

The invention provides an automatic software testing method and system based on artificial intelligence. The method comprises the following steps: analyzing a software demand document by using an NLP technology, converting the software demand document into structured data, extracting a key function point, and identifying a software function point; generating a test case covering all function points through a deep learning model in combination with historical test data training; automatically executing a test operation, analyzing a test result, analyzing a test log by using a machine learning algorithm, identifying and classifying potential defects by using a data mining and pattern recognition technology, and generating a repair suggestion report; and analyzing feedback data in the test process, dynamically adjusting a strategy according to a test result by using a reinforcement learning algorithm, and optimizing a test case and the strategy. According to the invention, intelligent analysis of test data, automatic generation of test cases and automatic execution of a test process are realized; flexible configuration and expansion can be carried out according to the actual condition of software; the test efficiency and quality are improved, and the test cost and manpower consumption are reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of software testing, and more particularly, to an automated software testing method and system based on artificial intelligence. Background Art

[0002] Modern software systems often contain a large number of functional modules, complex interaction logics, and highly customized requirements. The increase in software complexity makes traditional manual testing methods inefficient and error-prone, and it is difficult to ensure the comprehensiveness and accuracy of testing.

[0003] With the rapid expansion of the software industry and the acceleration of software iteration speed, the scale of software systems has expanded rapidly, and the testing work faces huge challenges in terms of testing efficiency and quality. How to complete high-quality testing within a limited time has become an urgent problem in the field of software testing. Although there are already various automated testing tools on the market, although these tools have improved testing efficiency to a certain extent, they often rely on preset rules and templates for testing, and it is difficult to flexibly handle various situations that may occur in actual development. There are still obvious deficiencies in dealing with complex logics, abnormal situations, and boundary condition testing. At the same time, the existing automated software testing technologies are costly in generating and maintaining test cases, and are often limited by specific test environments and resources, with great limitations. In addition, the high technical and personnel thresholds also limit the wide application of these automated testing tools.

[0004] In summary, with the development of the software industry, the requirements for software testing are getting higher and higher. Testing not only needs to cover all functional modules and interaction processes of the software, but also needs to be able to accurately identify and handle various abnormal situations. Therefore, it is necessary to meet the testing requirements of high efficiency and comprehensive coverage. Traditional testing methods and tools are no longer able to meet these requirements. Although there are many automated testing solutions emerging in the current market, and they have shown certain effects in improving testing efficiency and accuracy, however, in key areas such as deeply analyzing complex business logics, accurately capturing and simulating abnormal scenarios, and intelligently generating diverse test cases, these tools still face significant limitations. This limitation hinders their full integration into and efficient support for the rapid iteration cycle and highly personalized customization requirements pursued in modern software development. Summary of the Invention

[0005] In view of this, the purpose of the present invention is to propose an artificial intelligence-based automated software testing method and system. By introducing artificial intelligence technology and combining algorithms such as machine learning, natural language processing, or deep learning, it realizes the intelligentization of software testing, supports the full-process automated management from test case generation, execution to result analysis, and achieves intelligent analysis of test data, automatic generation of test cases, and automated execution of the test process. At the same time, it can also be flexibly configured and extended according to the actual situation of the software to meet the testing requirements of different software. It not only improves testing efficiency and quality but also reduces testing costs and labor consumption, providing strong support for the rapid development of the software industry.

[0006] The present invention provides an artificial intelligence-based automated software testing method, including the following steps:

[0007] S1. Use natural language processing (NLP) technology to parse and analyze the software requirement documents (such as user stories, functional specifications, etc.), convert the requirement documents into structured data, extract key function points and scenario descriptions, and automatically identify the function points of the software.

[0008] Preferably, based on Python, use libraries such as NLTK or Spacy for text processing.

[0009] S2. According to the requirement analysis results, train through a deep learning neural network (such as LSTM or Transformer) model in combination with historical test data to automatically generate high-quality test cases covering all function points.

[0010] Preferably, based on Python, use frameworks such as TensorFlow or PyTorch for model training.

[0011] Analyze software requirement documents, code, or user behavior through AI algorithms to intelligently generate test cases with high coverage and reduce manual intervention.

[0012] S3. According to the generated test cases, automatically execute the corresponding test operations, and the test operations include: user interface testing, API testing (covering multi-dimensional test scenarios such as user interface UI, function, and performance).

[0013] Preferably, based on Java, combine with the Selenium framework for Web testing, and based on Python for API testing.

[0014] Specifically, use automated testing tools such as Selenium and Appium to perform user interface (UI) testing, and use tools such as Postman for API testing.

[0015] S4. After the test is completed, analyze the test results. Use machine learning algorithms (such as decision trees, support vector machines, etc.) to analyze the test logs, and utilize data mining and pattern recognition techniques to predict potential defect areas and quickly locate the root cause of problems, automatically identify potential defects, classify the defects, and generate a repair suggestion report for developers' reference;

[0016] Preferably, based on Python, use tools such as scikit-learn or XGBoost for model training.

[0017] S5. Analyze the feedback data during the test process. Use reinforcement learning algorithms to dynamically adjust the strategy according to the test results, and automatically optimize the test cases and strategies to improve the test efficiency and accuracy.

[0018] Preferably, based on Python, use the Reinforcement Learning library (such as OpenAI Gym) for strategy optimization.

[0019] Furthermore, the method of using data mining and pattern recognition techniques in step S4 to predict potential defect areas and quickly locate the root cause of problems includes:

[0020] S41. Collect test case data based on multi-source data, and the multi-source data includes static data and dynamic data;

[0021] Preferably, the static data includes:

[0022] Software code: source code, version control records, annotation information.

[0023] Documents: requirement documents, design documents, and test reports.

[0024] The dynamic data includes:

[0025] Runtime logs: log files of program execution, error messages, stack traces.

[0026] Test results: test case execution results, code coverage, defect records.

[0027] Preferably, the test case data includes:

[0028] Execution history of test cases (pass rate, failure rate).

[0029] Covered code modules (test coverage).

[0030] Defect record data:

[0031] Frequency and distribution of defect occurrences (module / function dimension).

[0032] Severity and priority of defects (e.g., P1, P2).

[0033] Code change data:

[0034] Number of code modifications (commit frequency).

[0035] Time of the most recent modification.

[0036] Number of lines added or deleted in the code.

[0037] The present invention combines static code analysis and dynamic behavior learning techniques, improving the accuracy and effectiveness of test cases;

[0038] Clean the test case data, remove irrelevant fields (such as meaningless log lines, invalid code snippets), unify the format (e.g., standardize timestamps, make file paths consistent), and fill in missing values (e.g., missing test result fields can be inferred statistically);

[0039] Specifically, remove invalid or irrelevant data in the historical records, such as test cases with incomplete execution histories; merge duplicate data, such as different defect records for the same module.

[0040] Annotate the data according to the historical records and add data labels;

[0041] Preferably, the data labels include:

[0042] Positive samples: Clearly identify the locations of discovered defects.

[0043] Negative samples: Modules or paths that have been verified to have no problems;

[0044] S42. Extract static features, dynamic behavior features, test process features, and semantic features of the code;

[0045] Preferably, the static features include:

[0046] Code complexity: Cyclomatic Complexity, Halstead Metrics;

[0047] Code style: Number of lines, comment ratio, function length;

[0048] Dependency relationship: Call relationships between modules (through call graphs or control flow graphs).

[0049] The dynamic behavior features are extracted from runtime data and include:

[0050] Log analysis: Key error messages in the logs, call stack patterns;

[0051] Performance metrics: CPU usage, memory usage, I / O performance;

[0052] Path coverage: Record the actually executed paths through a code coverage tool.

[0053] The described test process characteristics include:

[0054] Use the information during the test execution process;

[0055] Test case pass rate;

[0056] Context of failed test cases (input, output, expected results).

[0057] The described semantic characteristics include:

[0058] Use natural language processing (NLP) techniques to analyze requirement documents or comments:

[0059] Keyword extraction (such as words like "error", "fail" in problem descriptions);

[0060] Text similarity calculation (such as semantic matching through TF-IDF or Word2Vec).

[0061] S43. Input the extracted features into a prediction model, perform data modeling, and construct a defect prediction model;

[0062] Specifically, the type of the defect prediction model can adopt any one of the following:

[0063] Supervised learning model: Use historical defect data as the training set. The model includes random forest, gradient boosting decision tree (GBDT), deep neural network (DNN), etc.;

[0064] Unsupervised learning model: In the case of lacking labels, use clustering algorithms (such as K-means) to identify abnormal patterns.

[0065] Semi-supervised learning: Combine a small amount of labeled data and a large amount of unlabeled data to improve the prediction effect.

[0066] Train and validate the defect prediction model, split the dataset into a training set, a validation set, and a test set to ensure the generalization ability of the model; Use precision, recall, and F1-score to evaluate the performance of the defect prediction model;

[0067] Precision: The accuracy of predicting as a defect;

[0068] Recall: The ability to discover all defects;

[0069] F1 score: comprehensively evaluate precision and recall;

[0070] S44. For newly submitted code or newly added functional modules, use the trained defect prediction model to predict the defect possibility, generate a defect possibility score for each line of code, each module, or each log; use a visualization tool to locate the defect; use pattern recognition technology to analyze the root cause of the defect.

[0071] Preferably, establish a mapping relationship between the module and the test case, record the functional modules affected by each test case, and add an associated historical defect record to each test case.

[0072] Preferably, automatically generate an easy-to-understand test report, including key metrics such as execution results, coverage rate, and defect distribution, and provide result analysis with real-time data visualization. Facilitate the team to understand the test progress and system stability.

[0073] Exemplarily, the processing flow of the automated software testing of the present invention in a practical application includes the following steps:

[0074] Assume that the test system discovers an abnormal log "NullPointerException":

[0075] Data analysis: Extract the time of the log, the module where it occurs, and its call stack.

[0076] Feature extraction: Analyze the code complexity of the abnormal module and the recent change records.

[0077] Model prediction: Use the defect prediction model to score and identify the most likely defective module.

[0078] Locate the defect: Combine the call chain diagram to locate the source of the exception as the recently modified function foo() in ModuleA.

[0079] The above processing process combines data mining (data collection, feature extraction, modeling) and pattern recognition (anomaly detection, pattern matching), and can realize intelligent prediction and accurate positioning of defects. Through this process, developers can predict potential risks before problems occur and quickly lock down the root cause of the problems, improving the reliability and maintenance efficiency of the system.

[0080] The present invention uses data mining and pattern recognition technologies to predict potential defect areas and quickly locate the root cause of problems. It can generate specific repair suggestions to assist developers in problem repair.

[0081] Furthermore, the method for dynamically adjusting the policy and automatically optimizing the test cases and policies in the S5 step includes:

[0082] Statistically analyze the defect data in the historical records, and calculate the defect occurrence ratio of each module: the frequency of defects. The calculation formula for the frequency of defects is:

[0083] ;

[0084] Specifically, the higher the defect occurrence frequency of a module, the higher the priority;

[0085] Calculate the severity weight of the defects, and adjust the priority based on the defect severity. The calculation formula for the severity weight of the defects is:

[0086] ;

[0087] Among them, is the severity weight of the defects;

[0088] is the frequency of defect occurrence;

[0089] Integrate the defect frequency and severity weight to determine the priority of the module.

[0090] Specifically, modules with severe defects require a higher priority;

[0091] Calculate the code change frequency of the module. The calculation formula is:

[0092] ;

[0093] According to the code change frequency, statistically analyze the commit data of the code repository by module and calculate the change weight;

[0094] Specifically, modules that are frequently modified may have more problems;

[0095] According to the module weight covered by the test cases, calculate the comprehensive priority score of the test cases. The priority calculation formula is:

[0096] ;

[0097] Among them, is the coverage degree of the test case for the module (for example, 1 means full coverage and 0 means no coverage).

[0098] Furthermore, the dynamic adjustment of the strategy in the S5 step, and the automatic optimization of the application process of test cases and strategies in actual testing include:

[0099] Initialize the test priority, execute the priority calculation formula, score all test cases, sort them according to the score, and generate a test priority list;

[0100] During the test execution process, new data is collected in real time (such as case failure rate, module coverage rate); the priority scores of test cases are updated, and the test order is dynamically adjusted;

[0101] Classify and grade test cases according to priority, divided into three categories: high, medium, and low;

[0102] Among them, the high priority is: covering high-risk modules or functions;

[0103] The medium priority is: general modules, executed according to the plan;

[0104] The low priority is: low execution frequency, can be delayed.

[0105] The optimization strategy of the present invention has the following advantages:

[0106] High efficiency: By preferentially executing high-risk test cases, potential defects can be quickly discovered. Reduce unnecessary redundant tests and save test resources.

[0107] Flexibility: Adjust the priority in real time to adapt to rapid iteration and dynamic requirement changes.

[0108] Accuracy: Based on the analysis of historical data, improve the accuracy of problem discovery.

[0109] Furthermore, the method of using pattern recognition technology to analyze the root cause of defects in the S44 step includes:

[0110] Locate the context where the problem occurs through the error stack and time series pattern in the log.

[0111] Use the dependency graph to analyze the interaction between modules and identify possible conflicts or anomalies.

[0112] Furthermore, the method of using a visualization tool to locate defects in the S44 step includes:

[0113] Use a graphical tool to display the prediction results;

[0114] Use a heat map to show the defect probability of code modules;

[0115] Display the functions or modules in the call chain that may cause problems.

[0116] The present invention analyzes the functional requirements document and historical test data of the software, and uses AI algorithms to automatically generate test cases covering all possible test scenarios. It can automatically execute test operations according to the generated test cases, and can handle various types of tests such as complex user interactions, interface tests, and API tests. Using AI technology to analyze the test results, detect defects, and can automatically classify and prioritize them to guide developers to make repairs. By continuously learning the data during the test execution process, it can automatically adjust the test strategy and cases, and gradually improve the test efficiency and accuracy.

[0117] The present invention also provides an artificial intelligence-based automated software testing system that executes the artificial intelligence-based automated software testing method as described above, including:

[0118] Requirement analysis module: used to parse and analyze the requirement document of the software using natural language processing (NLP) technology, convert the requirement document into structured data, extract key function points and scenario descriptions, and automatically identify the function points of the software;

[0119] AI test case generation module: used to train according to the requirement analysis results through a deep learning neural network model in combination with historical test data, and automatically generate high-quality test cases covering all function points;

[0120] Automatic test execution module: used to automatically execute corresponding test operations according to the generated test cases, and the test operations include: user interface test, API test;

[0121] Result analysis and defect detection module: used to analyze the test results, use machine learning algorithms, analyze the test logs, automatically identify potential defects, classify the defects, and generate a repair suggestion report;

[0122] Feedback and improvement module: used to analyze the feedback data during the test process, use reinforcement learning algorithms, dynamically adjust the strategy according to the test results, and automatically optimize the test cases and strategies.

[0123] Furthermore, the result analysis and defect detection module includes:

[0124] Data preparation unit: used to collect test case data based on multi-source data, and the multi-source data includes static data and dynamic data; clean the test case data, remove irrelevant fields, unify the format, and fill in missing values; label the data according to historical records and add data labels;

[0125] Feature extraction unit: used to extract static characteristics, dynamic behavior characteristics, test process characteristics, and semantic characteristics of the code;

[0126] Model training unit: used to input the extracted features into a prediction model, perform data modeling, and construct a defect prediction model; train and validate the defect prediction model, split the data set into a training set, a validation set, and a test set to ensure the generalization ability of the model; evaluate the performance of the defect prediction model using precision, recall, and F1 score.

[0127] Defect localization unit: used to predict the defect possibility of newly submitted code or newly added functional modules by using the trained defect prediction model, and generate a defect possibility score for each line of code, each module, or each log.

[0128] By introducing artificial intelligence technologies (such as machine learning and deep learning), the present invention significantly improves the processing ability of complex logic, enhances the test coverage of abnormal situations and boundary conditions, automatically generates and maintains test cases, reduces the dependence on specific environments and resources, and lowers the technical and personnel usage thresholds. Through these innovative designs, the efficiency and accuracy of software testing are improved, while the testing cost is reduced, providing strong support for the rapid development of the modern software industry.

[0129] The present invention also provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the steps of the automated software testing method based on artificial intelligence as described above are implemented.

[0130] The present invention also provides a computer device, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the steps of the automated software testing method based on artificial intelligence as described above are implemented.

[0131] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0132] The automated software testing method and system based on artificial intelligence provided by the present invention realize the intelligentization of software testing by introducing artificial intelligence technology and combining algorithms such as machine learning, natural language processing, or deep learning. It supports the full-process automated management from test case generation, execution to result analysis, and realizes the intelligent analysis of test data, the automatic generation of test cases, and the automated execution of the test process. Through the overall system architecture design, including the collaborative working methods of each module of the system, it automatically generates test cases from unstructured documents or code using natural language processing and machine learning technologies; uses reinforcement learning or deep learning technologies to dynamically adjust the test path; constructs a defect prediction model for predicting software defect areas; dynamically adjusts the test strategy based on historical data, improving the test efficiency; visualizes the test results in real time and automatically generates test reports. By introducing artificial intelligence technology and combining algorithms such as machine learning, natural language processing, or deep learning, it realizes the intelligentization of software testing, supports the full-process automated management from test case generation, execution to result analysis, and realizes the intelligent analysis of test data, the automatic generation of test cases, and the automated execution of the test process. At the same time, it also adapts to the compatibility of various software environments: it can support the adaptation technology of multiple programming languages and test frameworks, can be flexibly configured and extended according to the actual situation of the software, meets the test requirements of different software, and can ensure the application effect of the system in different development environments. It not only improves the test efficiency and quality, but also reduces the test cost and human consumption, providing strong support for the rapid development of the software industry. BRIEF DESCRIPTION OF THE DRAWINGS

[0133] By reading the detailed description of the preferred embodiments below, various other advantages and benefits will become clear to those of ordinary skill in the art. The drawings are only for the purpose of showing the preferred embodiments and are not considered to be a limitation of the present invention.

[0134] In the drawings:

[0135] Figure 1 is the overall architecture schematic diagram of the automated software testing system based on artificial intelligence according to the embodiment of the present invention;

[0136] Figure 2 is the detailed flowchart of the automatic execution test module according to the embodiment of the present invention;

[0137] Figure 3 is the flowchart of an automated software testing method based on artificial intelligence of the present invention;

[0138] Figure 4 is the flowchart of the method for predicting potential defect areas and quickly locating the root cause of problems using data mining and pattern recognition technologies of the present invention;

[0139] Figure 5Schematic diagram of the computer device according to an embodiment of the present invention. Detailed implementation manners

[0140] Here, exemplary embodiments will be described in detail, and examples thereof are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. On the contrary, they are merely examples of devices and products consistent with some aspects of the present disclosure as detailed in the appended claims.

[0141] The terms used in the present disclosure are for the purpose of describing particular embodiments only and are not intended to limit the present disclosure. The singular forms "a", "said", and "the" used in the present disclosure and the appended claims are also intended to include the plural forms unless the context clearly dictates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0142] It should be understood that although the terms first, second, third, etc. may be used in the present disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present disclosure, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "when" or "while" or "in response to determining".

[0143] The embodiments of the present invention will be further described in detail below.

[0144] An embodiment of the present invention provides an automated software testing method based on artificial intelligence. Refer to Figure 3 As shown, the method includes the following steps:

[0145] S1. Use natural language processing (NLP) technology to parse and analyze the requirement documents of the software (user stories, functional specifications), convert the requirement documents into structured data, extract key function points and scenario descriptions, and automatically identify the function points of the software;

[0146] S2. According to the requirement analysis results, train through a deep learning neural network (LSTM) model in combination with historical test data, and automatically generate high-quality test cases covering all function points;

[0147] Analyze the software requirement documents, code, or user behavior through an AI algorithm, and intelligently generate test cases with high coverage, reducing manual intervention;

[0148] S3. Automatically execute corresponding test operations according to the generated test cases. The test operations include: user interface testing and API testing;

[0149] Specifically, use automated testing tools such as Selenium and Appium to perform user interface (UI) testing, and use tools such as Postman for API testing;

[0150] S4. After the tests are executed, analyze the test results. Use machine learning algorithms (decision trees, support vector machines) to analyze the test logs, and utilize data mining and pattern recognition techniques to predict potential defect areas and quickly locate the root cause of problems, automatically identify potential defects, classify the defects, and generate a repair suggestion report;

[0151] For the method of using data mining and pattern recognition techniques to predict potential defect areas and quickly locate the root cause of problems, see Figure 4 as shown below, which includes the following steps:

[0152] S41. Collect test case data based on multi-source data. The multi-source data includes static data and dynamic data;

[0153] The static data includes:

[0154] Software code: source code, version control records, and comment information.

[0155] Documents: requirement documents, design documents, and test reports.

[0156] The dynamic data includes:

[0157] Runtime logs: log files of program execution, error messages, and stack traces.

[0158] Test results: test case execution results, code coverage, and defect records.

[0159] The test case data includes:

[0160] Execution history of test cases (pass rate, failure rate).

[0161] Covered code modules (test coverage).

[0162] Defect record data:

[0163] Frequency and distribution of defect occurrences (module / function dimension).

[0164] Severity and priority of defects (e.g., P1, P2).

[0165] Code change data:

[0166] The number of code modifications (commit frequency).

[0167] The most recent modification time.

[0168] The number of lines of code added or deleted.

[0169] Combining static code analysis and dynamic behavior learning techniques to improve the accuracy and effectiveness of test cases;

[0170] Clean the test case data, remove irrelevant fields (such as meaningless log lines, invalid code snippets), unify the format (e.g., standardize timestamps, make file paths consistent), and fill in missing values (e.g., missing test result fields can be inferred statistically);

[0171] In this embodiment, remove invalid or irrelevant data from the historical records, such as test cases with incomplete execution histories; merge duplicate data, such as different defect records for the same module.

[0172] Annotate the data according to the historical records and add data labels; the data labels include:

[0173] Positive samples: Clearly identify the locations of discovered defects.

[0174] Negative samples: Modules or paths that have been verified to have no problems;

[0175] S42. Extract the static characteristics, dynamic behavior characteristics, test process characteristics, and semantic characteristics of the code;

[0176] The static characteristics include:

[0177] Code complexity: Cyclomatic Complexity, Halstead Metrics;

[0178] Code style: Number of lines, comment ratio, function length;

[0179] Dependency relationship: The call relationship between modules (through a call graph or control flow graph).

[0180] The dynamic behavior characteristics are extracted from runtime data and include:

[0181] Log analysis: Key error messages in the log, call stack patterns;

[0182] Performance metrics: CPU usage, memory usage, I / O performance;

[0183] Path coverage: Record the actual executed paths through a code coverage tool.

[0184] The test process characteristics include:

[0185] Use the information during the test execution process;

[0186] The passing rate of test cases;

[0187] The context of failed test cases (input, output, expected results);

[0188] The semantic features include:

[0189] Use natural language processing (NLP) techniques to analyze requirement documents or comments;

[0190] Keyword extraction (such as words like "error", "fail" in problem descriptions);

[0191] Text similarity calculation (such as semantic matching through TF-IDF or Word2Vec).

[0192] S43. Input the extracted features into a prediction model for data modeling to construct a defect prediction model;

[0193] The defect prediction model of this embodiment adopts a supervised learning model: Use historical defect data as the training set, and the model adopts a deep neural network (DNN);

[0194] Train and validate the defect prediction model, divide the data set into a training set, a validation set, and a test set to ensure the generalization ability of the model; Use precision, recall, and F1-score to evaluate the performance of the defect prediction model;

[0195] Precision: The accuracy of predicting defects.

[0196] Recall: The ability to discover all defects.

[0197] F1-score: Comprehensively evaluate precision and recall;

[0198] S44. For newly submitted code or newly added functional modules, use the trained defect prediction model to predict the defect possibility, generate a defect possibility score for each line of code, each module, or each log; Use a visualization tool to locate the defects; Use pattern recognition techniques to analyze the root causes of defects.

[0199] Establish a mapping relationship between modules and test cases, record the functional modules affected by each test case, and add associated historical defect records to each test case.

[0200] Automatically generate an easy-to-understand test report, including key metrics such as execution results, coverage, and defect distribution, and provide real-time data visualization result analysis. Facilitate the team to understand the test progress and system stability.

[0201] The method for analyzing the root cause of defects using pattern recognition technology includes:

[0202] Locate the context where the problem occurs through the error stack and time series pattern in the log.

[0203] Use the dependency graph to analyze the interaction between modules and identify possible conflicts or anomalies.

[0204] The method for locating defects using visualization tools includes:

[0205] Use a graphical tool to display the prediction results;

[0206] Use a heat map to show the defect probability of code modules;

[0207] Show the functions or modules in the call chain that may cause problems.

[0208] This embodiment uses data mining and pattern recognition technologies to predict potential defect areas and quickly locate the root cause of problems, and can generate specific repair suggestions to assist developers in problem repair.

[0209] The above processing process combines data mining (data collection, feature extraction, modeling) and pattern recognition (anomaly detection, pattern matching), which can realize intelligent prediction and accurate positioning of defects. Developers can predict potential risks before problems occur and quickly lock the root cause of problems, improving the reliability and maintenance efficiency of the system.

[0210] S5. Analyze the feedback data during the test process, use the reinforcement learning algorithm, and dynamically adjust the strategy according to the test results to automatically optimize the test cases and strategies.

[0211] The method for dynamically adjusting the strategy and automatically optimizing the test cases and strategies includes:

[0212] Statistically analyze the defect data in the historical records, calculate the defect occurrence ratio of each module: the frequency of defects, and the calculation formula for the frequency of defects is:

[0213] ;

[0214] The higher the defect occurrence frequency of the module, the higher the priority;

[0215] Calculate the severity weight of the defect, and adjust the priority based on the defect severity. The calculation formula for the severity weight of the defect is:

[0216] ;

[0217] Among them, is the severity weight of the defect;

[0218] is the frequency of defect occurrence;

[0219] Determine the priority of the module by integrating the defect frequency and severity weight.

[0220] Modules with severe defects require a higher priority;

[0221] Calculate the code change frequency of the module. The calculation formula is:

[0222] ;

[0223] According to the code change frequency, count the commit data of the code repository by module and calculate the change weight;

[0224] Modules that are frequently modified may have more problems;

[0225] According to the module weight covered by the test cases, calculate the comprehensive priority score of the test cases. The priority calculation formula is:

[0226] ;

[0227] Among them, is the coverage degree of the test case for the module (1 means full coverage, 0 means no coverage).

[0228] The dynamic adjustment of the strategy and the automatic optimization of the application process of test cases and strategies in actual testing include:

[0229] Initialize the test priority, execute the priority calculation formula, score all test cases, sort them according to the score, and generate a test priority list;

[0230] During the test execution process, collect new data in real time (including: test case failure rate, module coverage rate); update the priority score of the test cases and dynamically adjust the test order;

[0231] Classify and grade the test cases according to the priority, and divide them into three categories: high, medium, and low;

[0232] Among them, the high priority is: covering high-risk modules or functions;

[0233] The medium priority is: general modules, executed according to the plan;

[0234] The low priority is: low execution frequency, can be delayed.

[0235] This embodiment includes the following modules and historical data:

[0236] Calculate the weights through formulas:

[0237] Defect frequency weight: Module A = = 0.33

[0238] Severity adjustment weight:

[0239] Change weight: Module A = 0.58 + = 0.93

[0240] After comprehensive scoring:

[0241] Test Case 1 = 0.93

[0242] Test Case 2 = 0.93

[0243] Test Case 3 = 0.50

[0244] Test Case 4 = 1.10

[0245] Test Case 5 = 1.10

[0246] Test order:

[0247] Execute first: Test Case 4, 5

[0248] Execute second: Test Case 1, 2

[0249] Execute later: Test Case 3

[0250] In this embodiment, by executing high-risk test cases first, potential defects can be quickly discovered, unnecessary redundant tests can be reduced, and test resources can be saved; the priority can be adjusted in real time to adapt to rapid iteration and dynamic requirement changes; based on the analysis of historical data, the accuracy of problem discovery is improved.

[0251] The present invention aims to solve several key problems faced by existing automated software testing technologies, including complex logic processing challenges, insufficient testing of exceptions and boundary conditions, high costs for test case generation and maintenance, strong dependence on environments and resources, and high technical and personnel thresholds. By introducing artificial intelligence technologies such as machine learning and deep learning, the present invention will achieve intelligent analysis and verification of complex logic, automatically generate diverse test cases for exceptions and boundary conditions, reduce the costs of test case generation and maintenance, optimize the operating environment and resource requirements of test tools, and provide a user-friendly interface and operation process to lower the technical and personnel usage thresholds. These innovations will significantly improve the efficiency and accuracy of software testing and provide strong support for the rapid development of the modern software industry.

[0252] An embodiment of the present invention also provides an automated software testing system based on artificial intelligence, which executes the automated software testing method based on artificial intelligence as described above, including:

[0253] A requirements analysis module: used to parse and analyze the requirements document of the software using natural language processing (NLP) technology, convert the requirements document into structured data, extract key function points and scenario descriptions, and automatically identify the function points of the software;

[0254] An AI test case generation module: used to train through a deep learning neural network model according to the requirements analysis results, combined with historical test data, and automatically generate high-quality test cases covering all function points;

[0255] An automatic test execution module: used to automatically execute corresponding test operations according to the generated test cases, and the test operations include: user interface testing, API testing;

[0256] A result analysis and defect detection module: used to analyze the test results, use machine learning algorithms to analyze the test logs, automatically identify potential defects, classify the defects, and generate a repair suggestion report;

[0257] A feedback and improvement module: used to analyze the feedback data during the test process, use reinforcement learning algorithms to dynamically adjust the strategy according to the test results, and automatically optimize the test cases and strategies.

[0258] The result analysis and defect detection module includes:

[0259] A data preparation unit: used to collect test case data based on multi-source data, where the multi-source data includes static data and dynamic data; clean the test case data, remove irrelevant fields, unify the format, and fill in missing values; label the data according to historical records and add data labels;

[0260] Feature extraction unit: used to extract the static features, dynamic behavior features, test process features, and semantic features of the code;

[0261] Model training unit: used to input the extracted features into a prediction model for data modeling to construct a defect prediction model; train and validate the defect prediction model, divide the data set into a training set, a validation set, and a test set to ensure the generalization ability of the model; evaluate the performance of the defect prediction model using precision, recall, and F1 score;

[0262] Defect localization unit: used to predict the defect possibility of newly submitted code or newly added functional modules using the trained defect prediction model, and generate defect possibility scores for each line of code, each module, or each log.

[0263] As Figure 1 shows the overall architecture of the artificial intelligence-based automated software testing system of this embodiment, including modules such as requirement analysis, AI test case generation, automatic execution, result analysis and defect detection, feedback and improvement, etc. The relationship between each module is represented by arrows, showing the data flow and control flow. Shows the detailed process of the requirement analysis module, including steps such as input of requirement documents, NLP processing, and function point extraction. Shows the working principle of the AI test case generation module, how to generate test cases through a deep learning model. Shows the relationship between the input data (requirements and historical data) and the output result (automatically generated test cases).

[0264] As Figure 2 shows the detailed process of the automatic execution test module of this embodiment, including the execution of various tests such as UI automation testing, API testing, and performance testing. Shows the specific process of test result analysis and defect detection, how to identify and classify defects through machine learning algorithms. Shows how the system optimizes the test strategy through reinforcement learning based on the test results and defect feedback to improve the test efficiency.

[0265] The embodiment of the present invention also provides a computer device, Figure 5 is a schematic structural diagram of a computer device provided by the embodiment of the present invention; see the attached drawings Figure 5 As shown, the computer device includes: an input system 23, an output system 24, a memory 22, and a processor 21; the memory 22 is used to store one or more programs; when the one or more programs are executed by the one or more processors 21, the one or more processors 21 implement the artificial intelligence-based automated software testing method provided in the above embodiment; where the input system 23, the output system 24, the memory 22, and the processor 21 can be connected through a bus or other means, Figure 5 taking the connection through the bus as an example.

[0266] The memory 22, as a computable device-readable and writable storage medium, can be used to store software programs and computer-executable programs, such as the program instructions corresponding to the artificial intelligence-based automated software testing method described in the embodiments of the present invention. The memory 22 may mainly include a program storage area and a data storage area. Among them, the program storage area may store an operating system and application programs required for at least one function. The data storage area may store data created according to the use of the device, etc. In addition, the memory 22 may include a high-speed random access memory and may also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage devices. In some instances, the memory 22 may further include a memory remotely set relative to the processor 21, and these remote memories may be connected to the device through a network. Examples of the above network include but are not limited to the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0267] The input system 23 can be used to receive input digital or character information and generate key signal inputs related to the user settings and function controls of the device. The output system 24 may include a display device such as a display screen.

[0268] The processor 21 executes various functional applications and data processing of the device by running the software programs, instructions, and modules stored in the memory 22, that is, implements the above-mentioned artificial intelligence-based automated software testing method.

[0269] The above-provided computer device can be used to execute the artificial intelligence-based automated software testing method provided in the above embodiments, and has corresponding functions and beneficial effects.

[0270] An embodiment of the present invention further provides a storage medium containing computer-executable instructions, which are used to execute the artificial intelligence-based automated software testing method provided in the above embodiment when executed by a computer processor. The storage medium is any of various types of memory devices or storage devices, including: installation media, such as CD-ROMs, floppy disks, or magnetic tape systems; computer system memories or random access memories, such as DRAM, DDRRAM, SRAM, EDO RAM, Rambus RAM, etc.; non-volatile memories, such as flash memories, magnetic media (such as hard disks or optical storage); registers or other similar types of memory elements, etc.; the storage medium may further include other types of memories or combinations thereof; additionally, the storage medium may be located in the first computer system in which the program is executed, or may be located in a different second computer system, and the second computer system is connected to the first computer system through a network (such as the Internet); the second computer system may provide program instructions to the first computer for execution. The storage medium includes two or more storage media that may reside in different locations (e.g., in different computer systems connected through a network). The storage medium may store program instructions (e.g., specifically implemented as a computer program) executable by one or more processors.

[0271] Of course, the computer-executable instructions of a storage medium containing computer-executable instructions provided by an embodiment of the present invention are not limited to the artificial intelligence-based automated software testing method described in the above embodiment, and may also execute related operations in the artificial intelligence-based automated software testing method provided by any embodiment of the present invention.

[0272] So far, the technical solution of the present invention has been described in conjunction with the preferred embodiments. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the protection scope of the present invention.

[0273] The above are only the preferred embodiments of the present invention and are not used to limit the present invention; for those skilled in the art, the present invention may have various changes and modifications. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An automated software testing method based on artificial intelligence, characterized in that: The following steps are involved: S1. Use natural language processing (NLP) technology to parse and analyze software requirement documents, convert requirement documents into structured data, extract key functional points and scenario descriptions, and automatically identify software functional points; S2. Based on the demand analysis results, the deep learning neural network model is trained in combination with historical test data to automatically generate high-quality test cases covering all functional points; S3. Automatically perform corresponding test operations according to the generated test cases, wherein the test operations include: user interface testing and API testing; S4. After the test is executed, the test results are analyzed, and machine learning algorithms are used to analyze the test logs. Data mining and pattern recognition techniques are used to predict potential defect areas and quickly locate the root causes of problems, automatically identify potential defects, classify the defects, and generate a repair suggestion report; S5. Analyze the feedback data during the test process, use reinforcement learning algorithms, dynamically adjust strategies based on test results, and automatically optimize test cases and strategies.

2. The method for automated software testing based on artificial intelligence according to claim 1, characterized in that: The method of using data mining and pattern recognition technology in step S4 to predict potential defect areas and quickly locate the root cause of the problem includes: S41, collecting test case data based on multi-source data, wherein the multi-source data includes static data and dynamic data; performing data cleaning on the test case data, removing irrelevant fields, unifying the format, and filling missing values; annotating the data according to historical records and adding data tags; S42, extract the static characteristics, dynamic behavior characteristics, test process characteristics and semantic characteristics of the code; S43, input the extracted features into the prediction model, perform data modeling, and construct a defect prediction model; train and verify the defect prediction model, divide the data set into a training set, a verification set, and a test set to ensure the generalization ability of the model; use precision, recall, and F1 score to evaluate the performance of the defect prediction model; S44. For newly submitted code or newly added functional modules, use the trained defect prediction model to predict the possibility of defects, and generate a defect possibility score for each line of code, each module or each log; use visualization tools to locate defects; and use pattern recognition technology to analyze the root cause of defects.

3. The artificial intelligence-based automated software testing method according to claim 1, characterized in that: The method of dynamically adjusting the strategy and automatically optimizing the test cases and strategies in step S5 includes: The defect data in the historical records are counted to calculate the defect occurrence ratio of each module: the defect frequency. The calculation formula for the defect frequency is: ； The severity weight of the defect is calculated, and the priority is adjusted based on the severity of the defect. The calculation formula of the severity weight of the defect is: ； in, is the severity weight of the defect; is the frequency of defect occurrence; Determine the priority of the module by combining defect frequency and severity weights; Calculate the code change frequency of the module, the calculation formula is: ； According to the code change frequency, the commit data of the code repository is counted by module and the change weight is calculated; According to the module weights covered by the test cases, the comprehensive priority score of the test cases is calculated. The priority calculation formula is: ； in, For test cases on modules coverage level.

4. The method for automated software testing based on artificial intelligence according to claim 3, characterized in that: The process of dynamically adjusting the strategy in step S5 and automatically optimizing the application of test cases and strategies in actual testing includes: Initialize the test priority, execute the priority calculation formula, score all test cases, sort them by score, and generate a test priority list; Collect new data in real time during test execution; update the priority scores of test cases and dynamically adjust the test order; Classify and grade the test cases according to their priority into three categories: high, medium, and low; Among them, the high priority is: covering high-risk modules or functions; Medium priority: general modules, executed as planned; Low priority means low execution frequency and can be delayed.

5. The artificial intelligence-based automated software testing method according to claim 2, characterized in that: The method of analyzing the root cause of the defect by using pattern recognition technology in step S44 includes: Locate the context of the problem through the error stack and time series patterns in the log; Use dependency graphs to analyze interactions between modules and identify possible conflicts or anomalies.

6. The method for automated software testing based on artificial intelligence according to claim 2, characterized in that: The method for locating defects using a visualization tool in step S44 includes: Use graphical tools to display forecast results; Use heatmaps to display the defect probability of code modules; Displays the functions or modules in the call chain that may be causing the problem.

7. An automated software testing system based on artificial intelligence, characterized in that: Executing the artificial intelligence-based automated software testing method according to any one of claims 1 to 6, comprising: Requirements analysis module: used to parse and analyze software requirements documents using natural language processing (NLP) technology, convert requirements documents into structured data, extract key functional points and scenario descriptions, and automatically identify software functional points; AI test case generation module: used to automatically generate high-quality test cases covering all functional points based on the demand analysis results, through deep learning neural network models, combined with historical test data for training; Automatic test execution module: used to automatically execute corresponding test operations according to the generated test cases, and the test operations include: user interface testing and API testing; Result analysis and defect detection module: used to analyze test results, use machine learning algorithms to analyze test logs, automatically identify potential defects, classify the defects, and generate repair suggestion reports; Feedback and Improvement Module: It is used to analyze the feedback data during the test process, use the reinforcement learning algorithm, dynamically adjust the strategy according to the test results, and automatically optimize the test cases and strategies.

8. The artificial intelligence-based automated software testing system according to claim 7, characterized in that: The result analysis and defect detection module includes: Data preparation unit: used to collect test case data based on multi-source data, the multi-source data includes static data and dynamic data; perform data cleaning on the test case data, remove irrelevant fields, unify the format, and fill in missing values; annotate data according to historical records and add data tags; Feature extraction unit: used to extract static characteristics, dynamic behavior characteristics, test process characteristics and semantic characteristics of the code; Model training unit: used to input the extracted features into the prediction model, perform data modeling, and build a defect prediction model; train and verify the defect prediction model, divide the data set into a training set, a validation set, and a test set to ensure the generalization ability of the model; use precision, recall, and F1 score to evaluate the performance of the defect prediction model; Defect location unit: used to predict the defect possibility of newly submitted code or newly added functional modules using the trained defect prediction model, and generate a defect possibility score for each line of code, each module or each log.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the artificial intelligence-based automated software testing method described in any one of claims 1 to 6 are implemented.

10. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the artificial intelligence-based automated software testing method as described in any one of claims 1 to 6 are implemented.

Images