Automatic testing method and device, electronic equipment and storage medium

Abstract

The invention provides an automatic testing method and device, electronic equipment and a storage medium, the method is applied to the field of software testing, and the method comprises the following steps: acquiring a test data set of software; training a software test model through a machine learning algorithm and the test data set; and automatically testing the software by using the trained software test model to obtain a test result. According to the method, the software test model can be trained through the machine learning algorithm and the test data set, and the software is automatically tested based on the trained software test model to obtain the test result, so that the operation is simple and convenient, the accuracy is relatively high, the test workload is reduced, the test timeliness and test efficiency are met, and the test time and cost are saved.

Description

Technical Field

[0001] The present application relates to the field of computers, and more specifically, to automated testing methods, devices, electronic devices, and storage media in the field of software testing. Background Art

[0002] With the rapid development of information technology, the scale of software applications has gradually expanded, and more and more manpower, material and financial resources have been invested in software research and development and testing. Various problems will inevitably arise in the process of software development. Therefore, software testing is performed to run or measure a software system to check whether it meets the specified requirements or to clarify the difference between the expected results and the actual results, so as to promptly discover abnormal conditions and deal with them.

[0003] In related technologies, software testing methods often require testers to manually enter test cases, manually execute tests, and manually evaluate test results. This method not only has a huge workload and low wage efficiency, but also makes it difficult to ensure test accuracy, wasting time and costs. Summary of the invention

[0004] The present application provides an automated testing method, device, electronic device and storage medium. The method can train a software testing model through a machine learning algorithm and a test data set, and perform automated testing on the software based on the trained software testing model to obtain test results. The operation is simple and convenient, the accuracy is high, the testing workload is reduced, the timeliness and efficiency of the test are met, and the testing time and cost are saved.

[0005] In a first aspect, an automated testing method is provided, the method comprising: obtaining a test data set for the software; training a software testing model using a machine learning algorithm and the test data set; and performing automated testing on the software using the trained software testing model to obtain a test result.

[0006] Through the above-mentioned technical scheme, the embodiment of the present application can train the software testing model through machine learning algorithms and test data sets, and perform automated testing on the software based on the trained software testing model to obtain test results. The operation is simple and convenient, the accuracy is high, the testing workload is reduced, the timeliness and efficiency of the test are met, and the testing time and cost are saved.

[0007] In combination with the first aspect, in some possible implementations, the training of the software testing model through the machine learning algorithm and the test data set includes: dividing the test data set into a training set and a test set; training the software testing model through the machine learning algorithm and the training set; and verifying the trained software testing model using the test set until a training stop condition is met, thereby obtaining a trained software testing model.

[0008] Through the above technical scheme, the embodiment of the present application can train the software testing model through machine learning algorithm and training set, verify the trained software testing model according to the test set, and if the verified software testing model meets the relevant conditions, the training is completed to improve the accuracy of the software testing model test.

[0009] In combination with the first aspect and the above-mentioned implementation methods, in some possible implementation methods, after training the software testing model through the machine learning algorithm and the test data set, it also includes: evaluating the performance and accuracy of the trained software testing model according to the model evaluation indicators; optimizing the model parameters and the machine learning algorithm according to the evaluation results of the software testing model.

[0010] Through the above technical scheme, the embodiment of the present application can evaluate the performance and accuracy of the trained software testing model according to the model evaluation indicators, and optimize the model parameters and machine learning algorithm according to the evaluation results, so as to improve the accuracy of the subsequently trained software testing model.

[0011] In combination with the first aspect and the above-mentioned implementation methods, in some possible implementation methods, obtaining a test data set for the software includes: obtaining test cases written manually or generated by a software testing tool; converting the test cases into a format recognizable by a machine learning algorithm, and using the machine learning algorithm to decompose the test cases into one or more feature values; and constructing the test data set based on the one or more feature values.

[0012] Through the above technical solution, the embodiment of the present application obtains test cases written manually or generated by software testing tools, converts them into a format recognizable by machine learning algorithms, and decomposes them into multiple feature values ​​to construct a data set, so as to facilitate subsequent training and verification of software testing models based on the data set.

[0013] In combination with the first aspect and the above-mentioned implementation methods, in some possible implementation methods, after the software is automatically tested using the trained software testing model to obtain the test results, it also includes: predicting the defects of the software based on the test results; and generating software optimization suggestions based on the type and frequency of the defects.

[0014] Through the above technical solution, the embodiment of the present application can predict software defects according to the test results, and generate optimization suggestions based on the type and frequency of software defects, so as to correct the software defects in a timely manner and achieve software optimization.

[0015] In combination with the first aspect and the above-mentioned implementation methods, in some possible implementation methods, predicting the defects of the software based on the test results also includes: collecting test data in the process of automated testing of the software by the software testing model; using data mining and machine learning algorithms to analyze the correlation features between different test data; and using the correlation features to analyze the defects of the software in the test results.

[0016] Through the above-mentioned technical scheme, the embodiment of the present application can collect test data of the software testing model during the automated testing of the software, use data mining and machine learning algorithms to analyze the correlation characteristics between different test data, and analyze the software defects in the test results based on the correlation characteristics, so as to facilitate the subsequent timely correction of software defects and achieve software optimization.

[0017] In combination with the first aspect and the above-mentioned implementation methods, in some possible implementation methods, generating optimization suggestions for the software based on the type and frequency of the defects also includes: if the software is the software of each subsystem in the software system, identifying the system relationship between the subsystems in the software system; generating optimization suggestions for the software system based on the type and frequency of the defects of each subsystem and the system relationship.

[0018] Through the above technical solution, the embodiment of the present application can identify the system relationship between each subsystem in the software system, and generate optimization suggestions for the software system according to the type, frequency and system relationship of the subsystem defects, so as to facilitate the subsequent timely correction of software defects and achieve software optimization.

[0019] In a second aspect, an automated testing device is provided, which includes: an acquisition module for acquiring a test data set for the software; a training module for training a software testing model through a machine learning algorithm and the test data set; and a testing module for performing automated testing on the software using the trained software testing model to obtain a test result.

[0020] In a third aspect, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the automated testing method as described in the above claims.

[0021] In a fourth aspect, a computer-readable storage medium is provided, which stores a computer program code. When the computer program code runs on a computer, the computer executes the method in the above-mentioned first aspect or any possible implementation manner of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 is a flow chart of the automated testing method provided in an embodiment of the present application;

[0023] Figure 2 It is an overall scheme diagram of the automated testing method provided in the embodiment of the present application;

[0024] Figure 3 It is a schematic diagram of the APP test provided in the embodiment of the present application;

[0025] Figure 4 It is a schematic diagram of the machine learning process provided by the embodiment of the present application;

[0026] Figure 5 is a feature extraction graph of the original data provided by the embodiment of the present application;

[0027] Figure 6 It is an overall schematic diagram of defect prediction provided by an embodiment of the present application;

[0028] Figure 7 It is a block diagram of the automated testing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0029] The technical solution in the present application will be described clearly and in detail below in conjunction with the accompanying drawings. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B: "and / or" in the text is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0030] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features.

[0031] Traditional testing methods can be divided into manual testing, automated testing, and static code analysis testing; among them, manual testing is one of the most basic software testing methods. Testers verify whether software functions meet requirements one by one by simulating actual usage scenarios. Although manual testing requires more time and manpower, it can find more software defects and problems. Automated testing uses testing tools and scripts to complete the software testing process. Compared with manual testing, automated testing is fast and efficient, which can save a lot of time and labor costs, but it requires the development of specialized testing tools and scripts, and has high testing skills requirements, and certain requirements for the skills of testers.

[0032] Static code analysis testing is a method for detecting software defects. It analyzes code and rules to find potential problems and can detect defects before software testing. Static code analysis can be performed manually or automatically. Compared with other testing methods, static code analysis is faster and more convenient, but it often requires expensive commercial tools and requires high coding skills of testers; therefore, the existing technology has the following problems:

[0033] 1. Inaccurate data collection: Existing technologies require the collection of historical data and other test data sources to generate the most relevant test cases. However, due to inaccurate data collection and lack of validity, the test cases and test result evaluations generated by machine learning algorithms may contain errors.

[0034] 2. The training time of machine learning models is long: the existing technology requires a large amount of data to be trained, so the machine learning model takes a long time to complete the training. This will increase the time and cost of the entire evaluation process, and due to the lack of sufficient real-time performance, it may not be able to meet some urgent testing tasks;

[0035] 3. Reliability issues of machine learning algorithms: Since the reliability of machine learning algorithms fluctuates due to issues such as data volume and data quality, errors are prone to occur in the test cases and test result evaluations generated by machine learning models. This will affect the accuracy of the test results and the application effect;

[0036] 4. Lack of effective result classification and defect prediction mechanism: The existing technology does not adopt an effective test result classification and defect prediction mechanism, which means that testers do not receive effective help when processing test results, resulting in reduced testing efficiency.

[0037] The automated testing method adopted in this application is more efficient and saves a lot of time and cost compared to the above-mentioned manual testing. Compared with the above-mentioned automated testing, it does not require specially developed testing tools and scripts, and is more convenient to operate. Compared with the above-mentioned static code analysis testing, it has lower costs and better testing effects.

[0038] Next, the application scenario or system architecture of the embodiment of the present application will be described.

[0039] Figure 1 It is a schematic flow chart of a vehicle control method provided in an embodiment of the present application.

[0040] For example, Figure 1 As shown, the method includes:

[0041] In step S101, a test data set of the software is obtained.

[0042] It is understandable that the embodiment of the present application obtains a test data set for the software to facilitate subsequent training of the software test model.

[0043] In an embodiment of the present application, obtaining a test data set for the software includes: obtaining test cases written manually or generated by a software testing tool; converting the test cases into a format recognizable by a machine learning algorithm, and using the machine learning algorithm to decompose the test cases into one or more eigenvalues; and constructing a test data set based on the one or more eigenvalues.

[0044] Among them, the machine learning algorithm can be a classification algorithm, a clustering algorithm, or a decision tree algorithm, without specific limitation.

[0045] It can be understood that the embodiments of the present application obtain test cases written manually or generated by software testing tools, and convert them into a format recognizable by a machine learning algorithm, use a machine learning algorithm to decompose the test cases into one or more eigenvalues, and construct a test data set based on the eigenvalues.

[0046] Specifically, a test case mainly includes four contents: use case title, prerequisites, test steps and expected results. Among them, the use case title mainly describes the test of a certain function, the prerequisites refer to the conditions that the use case title needs to meet, the test steps mainly describe the operation steps of the use case, and the expected results refer to meeting the expected requirements (development specifications, requirement documents, user requirements, etc.).

[0047] Machine learning algorithms may include: classification algorithms, clustering algorithms, decision tree algorithms, etc. Commonly used classification algorithms may include: Bayesian classification (a statistical classification method based on Bayes' theorem, which classifies by predicting the probability that a given tuple belongs to a specific class), K-nearest neighbor algorithm (using distance measurement to find the K data closest to the new data, and classify according to the K data types), linear classifier algorithm (classification by establishing a linear model), neural network algorithm (training to recognize patterns and classify by simulating biological neural networks), etc.; commonly used clustering analysis algorithms can be divided into partitioning method, hierarchical method, density-based method, grid-based method, model-based method; decision tree algorithm (divide the data set into different types and classify using a tree structure).

[0048] In step S102, a software testing model is trained using a machine learning algorithm and a test data set.

[0049] It is understandable that the embodiments of the present application can train a software testing model through a machine learning algorithm and a test data set, so as to facilitate the subsequent automated testing of the software according to the software testing model to obtain test results.

[0050] In an embodiment of the present application, a software testing model is trained by a machine learning algorithm and a test data set, including: dividing the test data set into a training set and a test set; training the software testing model by a machine learning algorithm and the training set; and verifying the trained software testing model using the test set until a training stop condition is met, thereby obtaining a trained software testing model.

[0051] Among them, the training stop condition can be that the accuracy of the verified software test model reaches a high level, such as the accuracy reaches 90% or 95%, etc., without specific limitation.

[0052] It can be understood that the embodiments of the present application can train the software testing model through a machine learning algorithm and a training set, and verify the trained software testing model according to the test set. If the verified software testing model meets the relevant conditions, the training is completed to improve the accuracy of the software testing model testing.

[0053] In an embodiment of the present application, after training the software testing model through a machine learning algorithm and a test data set, it also includes: evaluating the performance and accuracy of the trained software testing model according to model evaluation indicators; and optimizing model parameters and machine learning algorithms according to the evaluation results of the software testing model.

[0054] It can be understood that the embodiments of the present application can evaluate the performance and accuracy of the trained software testing model according to the model evaluation indicators, so as to optimize the model parameters and machine learning algorithms according to the evaluation results of the software testing model.

[0055] Specifically, model evaluation indicators may include precision, recall, ROC curve, F1 value and other indicators, among which precision is the ratio of the number of samples correctly predicted by the model to the total number of samples. High accuracy means that the model's prediction results are relatively accurate; recall refers to the proportion of all truly positive samples that are predicted as positive by the model; F1 value can be the harmonic mean of precision and recall, and its value range is generally between 0-1, where 1 indicates the best model performance and 0 indicates the worst model performance; ROC curve is a curve that plots the performance of a classification model. The horizontal axis indicates the false positive rate and the vertical axis indicates the true positive rate. The closer the ROC curve is to the upper left corner, the better the model performance. AUC is the area under the ROC curve. The larger the AUC value, the better the model performance.

[0056] In step S103, the trained software testing model is used to perform automated testing on the software to obtain test results.

[0057] It can be understood that the embodiment of the present application uses a trained software testing model to perform automated testing on the software to obtain test results. The operation is simple and convenient, the accuracy is high, the testing workload is reduced, the timeliness and efficiency of the test are met, and the testing time and cost are saved.

[0058] In an embodiment of the present application, after the software is automatically tested using the trained software testing model to obtain the test results, it also includes: predicting software defects based on the test results; and generating software optimization suggestions based on the type and frequency of the defects.

[0059] It is understandable that the embodiments of the present application can predict software defects based on test results, and generate optimization suggestions based on the type and frequency of software defects, so as to correct software defects in a timely manner and achieve software optimization.

[0060] Specifically, the types of software defects include: (1) Syntax defects: Syntax defects refer to defects caused by the program not conforming to the grammatical rules of the programming language; (2) Semantic defects: Semantic defects refer to defects caused by the program not conforming to the semantic analysis program requirements of the computer environment. Common semantic defects include type checking defects and execution restriction defects; (3) Operational defects: Operational defects refer to defects that occur during the actual operation of the program; (4) Description defects: Description defects refer to defects caused by the inconsistency between the requirement description and the user's stated requirements, or the inconsistency between the user's stated requirements and the user's actual requirements. Description defects are divided into three categories: incomplete description, inconsistent description, and ambiguous description; (5) Performance defects: Performance defects refer to the difference between the actual performance of the program and the required performance. The performance of the program can generally be measured in the following aspects: response time, running time, storage space, workspace requirements, etc.

[0061] For example, if the system collects the frequently occurring defect type of a software project, namely null pointer exception, then based on the collected data, the software testing model of the artificial intelligence system can generate the following optimization suggestions after training:

[0062] 1. Introduce null pointer detection tools: It is recommended to use static code analysis tools or IDE (Integrated Development Environment) plug-ins during the software development process to help developers detect and fix potential null pointer exceptions.

[0063] 2. Strengthen coding standards: It is recommended to promote coding standards in the team, especially for situations where null pointer exceptions are prone to occur, and strengthen compliance with relevant standards, such as performing non-null judgment before using the object.

[0064] 3. Strengthen unit testing: It is recommended that developers write comprehensive unit test cases to cover code paths that may cause null pointer exceptions, so as to detect and fix potential problems in a timely manner.

[0065] 4. Introduce automated testing tools: It is recommended to use automated testing tools, such as Selenium or Appium, to simulate user interactions and test system boundary conditions to detect potential null pointer exceptions.

[0066] 5. Code review and knowledge sharing: Encourage development teams to conduct code reviews and share knowledge to promote mutual learning among team members and to jointly understand and avoid potential codes that may cause null pointer exceptions.

[0067] These optimization suggestions can help the development team reduce the frequency of null pointer exceptions during project development and improve software reliability and stability; however, it should be noted that specific optimization strategies still need to be evaluated and adjusted based on project requirements and the actual development environment.

[0068] In an embodiment of the present application, predicting software defects based on test results also includes: collecting test data during the automated testing of the software by a software testing model; analyzing the correlation features between different test data using data mining and machine learning algorithms; and analyzing software defects in the test results using the correlation features.

[0069] It can be understood that the embodiments of the present application can collect test data from the process of automated testing of software by the software testing model, use data mining and machine learning algorithms to analyze the correlation characteristics between different test data, and analyze the software defects in the test results based on the correlation characteristics, so as to facilitate the subsequent timely correction of software defects and achieve software optimization.

[0070] It should be noted that the main tasks of data mining can be classification, clustering, association analysis, prediction, time series pattern and deviation analysis, etc., without specific limitation.

[0071] Specifically, when a particular input value or sequence of operations is directly related to two functions in the system, they are said to have a feature connection. Suppose you are testing a login system and find the following two problems:

[0072] 1. In the password input box, the system crashes when the user enters more than 8 characters.

[0073] 2. When the user enters an incorrect username, the system will display a "Login Failed" error message, but will not clear the password input box.

[0074] In this example, the feature association means that the length limit of the password input box caused the system to crash, while the incorrect username caused the error message display problem. In this case, there is a correlation between the specific features (password length limit and error message display) and the software defect (system crash and error message display problem). By discovering this feature association, developers can more accurately locate the root cause of the problem and make corresponding fixes. For example, solving the password input box length limit may be the key to solving the system crash problem, while fixing the error message display problem may require more careful handling of the error message generation and the logic of clearing the password input box.

[0075] In an embodiment of the present application, optimization suggestions for the software are generated based on the type and frequency of defects, and the method also includes: if the software is the software of each subsystem in the software system, identifying the system relationship between the subsystems in the software system; and generating optimization suggestions for the software system based on the type and frequency of defects in each subsystem and the system relationship.

[0076] It can be understood that the embodiments of the present application can identify the system relationship between each subsystem in the software system, and generate optimization suggestions for the software system based on the type, frequency and system relationship of the subsystem defects, so as to facilitate the subsequent timely correction of software defects and achieve software optimization.

[0077] Specifically, since software systems are now composed of multiple subsystems that are interdependent, it is difficult to test them using traditional methods. To deal with complex software system testing, the following methods can be used:

[0078] 1. Develop a test plan: Before testing begins, develop a detailed test plan. The plan should include test objectives, scope, resource allocation, test strategy, test methods, and schedule. This will help ensure that the testing work is carried out in an orderly manner and better cope with the complexity of the system.

[0079] 2. Analyze requirements and design documents: Carefully analyze the requirements and design documents of the software system in order to understand the overall architecture and functionality of the system. This will help identify key functions and modules and provide direction and focus for testing.

[0080] 3. Use appropriate testing methods: Select appropriate testing methods based on the nature and requirements of the system. Common testing methods include black box testing, white box testing, functional testing, performance testing, security testing, etc. Combined with specific test objectives and system characteristics, formulate corresponding testing strategies and test cases.

[0081] 4. Automated testing: Use automated testing tools to enhance testing efficiency and accuracy. Automated testing can help quickly execute a large number of test cases, reduce human errors, and improve the reusability and maintainability of testing.

[0082] 5. Phased testing and continuous integration: Divide testing into multiple phases and integrate and test them in a timely manner. Testing at each phase can verify different aspects of the system, ensuring that each module and function is fully tested and verified. Continuous integration allows testing work to be carried out in parallel with development work, and problems can be discovered and solved in a timely manner.

[0083] 6. Introduce gray box testing: In complex systems, gray box testing can combine the advantages of black box and white box testing to provide more comprehensive and in-depth test coverage. Through gray box testing, we can better understand the internal working principles of the system and discover hidden problems and vulnerabilities.

[0084] 7. Good communication and cooperation: Maintain good communication and cooperation with the development team, requirement team and other relevant parties, and share test results and problems found in a timely manner, which will help speed up problem solving and reduce the transmission of errors and omission of vulnerabilities.

[0085] In short, dealing with complex software system testing requires the comprehensive use of a variety of methods and techniques. By developing detailed plans, phased testing, automated testing, and flexible testing strategies, you can improve testing efficiency and quality, and discover potential problems and risks. At the same time, good communication and cooperation are also key factors to ensure the smooth progress of testing.

[0086] In summary, the automated testing method proposed in this application trains a software testing model through a machine learning algorithm and a test data set, and performs automated testing on the software based on the trained software testing model to obtain test results. It is simple and convenient to operate, has high accuracy, reduces the testing workload, meets the timeliness and efficiency of the test, and saves testing time and cost.

[0087] The following is a detailed description of the automated testing method. The overall solution predicts software defects based on artificial intelligence and machine learning and intelligently gives adjustment and optimization suggestions based on the prediction results, such as Figure 2 As shown, the details are as follows:

[0088] 1. Model building stage

[0089] (1) Collect and select test cases based on actual scenarios and usage conditions; or extract test cases from the software history database; these test cases can be manually written or generated by other software testing tools.

[0090] (2) Extract program modules from test cases and mark them with metrics, and then classify the measured program modules to generate test data. Metric marking can help evaluate quality, guide development, improve maintainability, reduce risks, and support decision-making.

[0091] (3) Preprocess the test data (such as Figure 3 As shown in the figure): The test data is converted into a machine-recognizable form, and the data is decomposed into individual feature values ​​using a machine learning algorithm. After the test data is characterized, it can be used to construct a test data set.

[0092] The test data set is divided into a training set and a test set. The model is trained on the training set and verified on the test set. Different model designs and algorithms can be used according to business needs.

[0093] It should be noted that if Figure 4 As shown, machine learning algorithms are mainly implemented through the front-end and back-end, among which the front-end is mainly visual operations, and the back-end is basically code logic, data storage, data flow, etc.

[0094] (4) Use machine learning algorithms to train software testing models. Commonly used machine learning algorithms include classification algorithms, clustering algorithms, decision tree algorithms, etc.

[0095] (5) The trained software test model is evaluated to determine the performance and accuracy of the model. Model evaluation indicators include accuracy, recall rate, precision, F1 value, etc., and the decision on whether to optimize the model parameters and algorithms is made based on the evaluation results.

[0096] It should be noted that if the results of the evaluation model show low accuracy, it is necessary to optimize and adjust the model parameters to improve the performance and generalization ability of the model. The process of optimizing the model involves data preprocessing, model selection, hyperparameter adjustment, training process management and other aspects. The specific steps are as follows:

[0097] (1) Data preprocessing: First, the data is preprocessed, including data cleaning, normalization, and standardization, to reduce noise and redundant information in the data and improve the training effect of the model.

[0098] (2) Model construction: Select a deep learning model suitable for the task, such as convolutional neural network (CNN), recurrent neural network (RNN), or Transformer. Select the appropriate model structure and number of layers based on the complexity of the problem and the characteristics of the dataset.

[0099] (3) Select the loss function: Select an appropriate loss function according to the characteristics of the task, such as mean square error (MSE), cross-entropy loss, etc. The loss function reflects the difference between the model prediction result and the true label.

[0100] (4) Select an optimization algorithm: Select an optimization algorithm that is suitable for the task, such as stochastic gradient descent (SGD), Adam, RMSprop, etc. The optimization algorithm is used to adjust the parameters in the model to minimize the loss function.

[0101] (5) Setting hyperparameters: Hyperparameters are configuration parameters of the model, such as learning rate, batch size, regularization parameters, etc. Use methods such as cross-validation or grid search to try different hyperparameter combinations and select the best combination.

[0102] (6) Training model: Use the training data set to train the model and update the model parameters through the back-propagation algorithm. Monitor the model training progress based on the performance of the training set and validation set to avoid overfitting or underfitting.

[0103] (7) Model evaluation: Use the test data set to evaluate the trained model and calculate the model's accuracy, precision, recall and other indicators. Adjust the model and hyperparameters based on the evaluation results.

[0104] (8) Parameter adjustment: Adjust the hyperparameters based on the evaluation results of the model. You can use grid search, random search, Bayesian optimization and other methods to search for the optimal hyperparameter combination.

[0105] (9) Model regularization: During the model training process, regularization methods such as L1 regularization, L2 regularization, or Dropout can be used to reduce the complexity of the model and improve the generalization ability.

[0106] (10) Model integration: By integrating the prediction results of multiple models, the performance and stability of the model can be improved. Common integration methods include voting method, averaging method, stacking method, etc.

[0107] 2. Model application stage

[0108] Apply the trained software testing model to actual software testing. By verifying the test results, the test dataset and model can be further optimized and improved.

[0109] Specifically, the new software data is input into the software testing model, and the test results are output. If the test results show that the software is correct, it means that there are no defects. If the test results show that there are abnormalities, the test data is input into the defect prediction model (such as Figure 5 ), output the specific types of defects, such as defects in the design of the quality element, defects in the model construction method, and defects related to the data set.

[0110] 3. Optimization suggestions

[0111] Machine learning can predict software defects based on test results and adaptively adjust test cases and test processes based on the type and frequency of software defects.

[0112] The factors affecting software defect prediction mostly include the following three types: the design of quality elements, the model construction method, and related issues of the data set.

[0113] (1) Design of quality element: Quality element is one of the key factors in software defect prediction. It is used to represent the characteristics of software code. The design of quality element should consider aspects related to software defects, such as code complexity, code coupling, and annotation quality. A well-designed quality element can more accurately capture the characteristics of software defects and improve the performance of the prediction model.

[0114] (2) Model construction method: The model construction method determines the performance and accuracy of the prediction model. Different construction methods may be suitable for different data and problems. For example, machine learning-based methods can use classification algorithms or regression algorithms to build prediction models, while statistical methods can use time series analysis or Bayesian networks. Choosing a suitable construction method can improve the prediction ability of the model.

[0115] (3) Dataset-related issues: Datasets are the basis for training and evaluating prediction models. The size, quality, and representativeness of the dataset will affect the effectiveness of the prediction model. If the dataset is too small, it may cause the model to overfit or underfit; if the dataset is not representative, the model may not be able to adapt to real scenarios; if the dataset has defects, it may introduce noise and affect the accuracy of the model. Therefore, correctly selecting and processing the dataset is an important step in software defect prediction.

[0116] The following will be described in conjunction with specific embodiments. Figure 6 As shown, the details are as follows:

[0117] Step 1: Collect training data and generate training data set;

[0118] Step 2, establishing a SVM (Support Vector Machine) model, and using the training data set to train the SVM model;

[0119] Step 3, verifying the trained SVM model;

[0120] Step 3: Start the APP or record the screen, analyze the video during the APP startup process, and generate a test data set;

[0121] Step 4, input the test data into the SVM model and output the test results;

[0122] Step 5: Identify the home page frame based on the test results to calculate the startup time.

[0123] In summary, the embodiments of the present application can make the software more automated based on machine learning software testing, especially using machine learning to implement adaptive testing, which can automatically identify problems that occur during the test process and make appropriate adjustments, greatly reducing the workload and time of testers; machine learning can discover potential errors and defects in the system through a large amount of data and analysis, and can adjust and improve the software according to the type and frequency of defects; software testing based on machine learning can collect and analyze a large amount of test data, and through data mining and machine learning algorithms, discover the associations and rules between test data, so as to realize intelligent analysis and judgment of test results; many modern software systems are composed of multiple subsystems, which are interdependent and difficult to test using traditional methods. Software testing based on machine learning can better cope with such complex software systems, and repair and adjust problems in the system by intelligently analyzing and testing the relationship between subsystems.

[0124] Next, the automated testing device proposed according to the embodiment of the present application is described with reference to the accompanying drawings.

[0125] Figure 7 It is a block diagram of an automated testing device according to an embodiment of the present application.

[0126] like Figure 7 As shown, the automated testing device 10 includes: an acquisition module 100 , a training module 200 and a testing module 300 .

[0127] Among them, the acquisition module 100 is used to obtain a test data set for the software; the training module 200 is used to train a software test model through a machine learning algorithm and the test data set; and the testing module 300 is used to perform automated testing on the software using the trained software test model to obtain test results.

[0128] It should be noted that the above explanations of the embodiment of the automated testing method are also applicable to the automated testing device of this embodiment, and will not be repeated here.

[0129] According to the automated testing device proposed in the embodiment of the present application, a software testing model is trained through a machine learning algorithm and a test data set, and the software is automatically tested based on the trained software testing model to obtain test results. The operation is simple and convenient, the accuracy is high, the testing workload is reduced, the timeliness and efficiency of the test are met, and the testing time and cost are saved.

[0130] In addition, an embodiment of the present application also protects an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the program to implement an automated testing method as described in the above embodiment.

[0131] This embodiment also provides a computer-readable storage medium, in which a computer program code is stored. When the computer program code is executed on a computer, the computer executes the above-mentioned related method steps to implement an automated testing method provided by the above-mentioned embodiment.

[0132] Among them, the device, computer-readable storage medium or chip provided in this embodiment is used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above, and will not be repeated here.

[0133] Through the description of the above implementation methods, technical personnel in the relevant field can understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0134] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0135] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. An automated testing method, characterized in that: The method comprises: Get a test dataset for the software; Training a software testing model using a machine learning algorithm and the test data set; The trained software testing model is used to automatically test the software and obtain the test results.

2. The method according to claim 1, characterized in that The method of training the software testing model by using the machine learning algorithm and the test data set includes: Dividing the test data set into a training set and a test set; Training the software testing model using a machine learning algorithm and the training set; The trained software test model is verified using the test set until a training stop condition is met, thereby obtaining a trained software test model.

3. The method according to claim 1 or 2, characterized in that: After training the software test model through the machine learning algorithm and the test data set, it also includes: Evaluate the performance and accuracy of the trained software testing model according to the model evaluation index; The model parameters and the machine learning algorithm are optimized according to the evaluation results of the software testing model.

4. The method according to claim 1, characterized in that The step of obtaining a test data set of the software includes: Obtain test cases written manually or generated by software testing tools; Converting the test case into a format recognizable by a machine learning algorithm, and decomposing the test case into one or more feature values ​​using the machine learning algorithm; The test data set is constructed according to the one or more feature values.

5. The method according to claim 1, characterized in that: After using the trained software testing model to automatically test the software and obtain the test results, it also includes: Predicting defects of the software according to the test results; Generate optimization recommendations for the software based on the type and frequency of the defects described.

6. The method according to claim 5, characterized in that The predicting the defects of the software according to the test results further includes: Collecting test data from the process of automated testing of software by the software testing model; Use data mining and machine learning algorithms to analyze the correlation characteristics between different test data; The associated features are used to analyze software defects in the test results.

7. The method according to claim 5, characterized in that Generating software optimization suggestions according to the types and frequencies of the defects further includes: If the software is software for each subsystem in the software system, identifying the system relationship between the subsystems in the software system; An optimization suggestion for the software system is generated based on the type and frequency of defects of each subsystem and the system relationship.

8. An automated testing device, characterized in that: The device comprises: An acquisition module is used to obtain a test data set for the software; A training module, used for training a software testing model through a machine learning algorithm and the test data set; The testing module is used to use the trained software testing model to perform automated testing on the software and obtain test results.

9. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the automated testing method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed, the automated testing method according to any one of claims 1 to 7 is implemented.

Images