Software testing method, platform and equipment

Abstract

The invention relates to the technical field of software testing, in particular to a software testing method, device and equipment. According to the method, the test priority of each module of a to-be-tested object is identified, a plurality of test cases are matched with each module according to the test priority, different test weights are allocated to each test case, and then the execution paths of the test cases are crossed and mutated according to the test weights and the test priority through a genetic algorithm, so that the test efficiency of the to-be-tested object is improved. And determining an optimal test case execution path, performing test defect feedback in real time according to a test result through a reinforcement learning algorithm in the test process, and adjusting the test priority of each module of the to-be-tested object and the test weight of the test case through a reward function according to a defect feedback result. Therefore, the software test platform can adaptively adjust the test execution path in the test process according to the test result, the test efficiency is greatly improved, and the defect discovery rate is also improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of software testing technology, and specifically to a software testing method, apparatus and device, and a computer-readable storage medium. Background Art

[0002] Software testing plays an important role in improving software quality, reducing costs and risks, enhancing user satisfaction, improving development efficiency and improving software life cycle management. This makes software testing an indispensable key link in any software project. During the software testing process, a test system is required to assist the software in functional testing.

[0003] The combination of artificial intelligence (AI) technology and software testing technology is leading to major changes in the field of software testing. AI technology, especially machine learning and deep learning, has begun to be applied in many aspects of software testing. Combining artificial intelligence technology with software testing technology can greatly improve the intelligence level of software testing and improve the efficiency of software testing.

[0004] The inventors of this application have discovered during their research that the existing technology combining artificial intelligence with software testing has many applications in the automatic generation of test cases and the automated execution of test processes. However, it lacks a process for adaptively and dynamically adjusting the test strategy during the test execution process. It is impossible to dynamically adjust the test case execution path according to the test results like manually executing test cases, resulting in poor testing results. Summary of the invention

[0005] In view of the above problems, the embodiments of the present application provide a software testing method, platform, device and computer-readable storage medium to solve the above technical problems existing in the prior art.

[0006] In one aspect of an embodiment of the present application, a software testing method is provided, the method comprising:

[0007] Get multiple sets of test cases and information about the objects to be tested;

[0008] Identify the test priority of each module of the object to be tested according to the information of the object to be tested;

[0009] Performing correlation matching between the plurality of groups of the test cases and the modules, determining the correlation between the test cases and the modules, and generating a test weight corresponding to the test cases according to the test priority and the correlation of the modules;

[0010] According to the test weight and the test priority of each module, performing crossover operation and mutation operation on the execution path of the test case by using a genetic algorithm to determine the optimal execution path of each test case;

[0011] Execute the test case according to the optimal execution path, and obtain the test result corresponding to the test case;

[0012] A reward function is determined according to the test result through a reinforcement learning algorithm, and the test priority of each module of the object to be tested and the test weight of the test case are adjusted according to the reward function.

[0013] Preferably, in some embodiments, before obtaining the multiple groups of test cases, the method further comprises:

[0014] Obtaining the requirement document information of the object to be tested;

[0015] Performing word segmentation and dependency syntactic analysis on the requirement document information to extract key semantic features;

[0016] The key semantic features are analyzed by a logistic regression model to generate multiple groups of test cases.

[0017] Preferably, in some embodiments, the information of the object to be tested includes code complexity, historical defect rate, code change frequency and business criticality;

[0018] The step of identifying the test priority of each module of the object to be tested according to the information of the object to be tested includes:

[0019] Through a preset risk assessment model, risk assessment is performed on each module of the object to be tested according to the code complexity, historical defect rate, code change frequency and business criticality;

[0020] The test priority corresponding to each module is determined according to the risk assessment result.

[0021] Preferably, in some embodiments, the step of identifying the test priority of each module of the object to be tested according to the information of the object to be tested includes:

[0022] In response to the operation on each module of the object to be tested, the test priority of each module input by the user is received.

[0023] Preferably, in some embodiments, after generating the test weights corresponding to the test cases according to the test priorities of the modules, the method further comprises:

[0024] Get historical test log data;

[0025] Training a defect prediction model based on the historical test log data;

[0026] Defect prediction is performed on the object to be tested according to the trained defect prediction model, and the test weight of the test case is adjusted according to the defect prediction result.

[0027] Preferably, in some embodiments, performing crossover and mutation operations on the execution paths of the test cases by a genetic algorithm according to the test weights and the test priorities of the modules to determine the optimal execution paths of the test cases includes:

[0028] Randomly select execution paths of multiple test cases from multiple groups of test cases as an initial population, wherein each execution path of the test case is an individual;

[0029] Determine a fitness function according to the historical defect rate of the object to be tested, the test weight of the test case and the test priority of each module of the object to be tested;

[0030] The fitness corresponding to each individual of the initial population is determined according to the fitness function, and a crossover operation and a mutation operation are performed on each individual according to the fitness to generate an optimal execution path for each test case until a preset termination condition is met.

[0031] Preferably, in some embodiments, performing crossover and mutation operations on the individuals according to the fitness to generate the optimal execution path for each of the test cases includes:

[0032] A crossover probability and a mutation probability are respectively allocated to each of the individuals according to the fitness, and after performing a crossover operation and a mutation operation on the initial population according to the crossover probability and the mutation probability, an optimal execution path of the test case is generated.

[0033] Preferably, in some embodiments, the crossover probability and mutation probability are respectively allocated to each of the individuals according to the fitness, specifically:

[0034] If the fitness of the individual is greater than or equal to a preset first threshold, assigning a first crossover probability and a first mutation probability to the individual according to the maximum fitness value in the population and the average fitness value in the population;

[0035] If the fitness of the individual is less than a preset first threshold, assigning a second crossover probability and a second mutation probability to the individual according to a preset constant;

[0036] The first crossover probability is smaller than the second crossover probability, the first mutation probability is smaller than the second mutation probability, and the preset first threshold is the average fitness value of the population.

[0037] Another aspect of the embodiments of the present application further provides a software testing platform, the platform comprising:

[0038] Data acquisition module: used to acquire multiple groups of test cases and information of the object to be tested, and identify the test priority of each module of the object to be tested according to the information of the object to be tested;

[0039] Data optimization module: matching the plurality of groups of test cases with the modules, determining the degree of association between the test cases and the modules, and generating the test weights corresponding to the test cases according to the test priorities of the modules and the degree of association; and performing crossover and mutation operations on the execution paths of the test cases through a genetic algorithm according to the test weights and the test priorities of the modules, so as to determine the optimal execution path of each test case;

[0040] Test execution module: used to execute the test case according to the optimal execution path and obtain the test result corresponding to the test case;

[0041] Defect feedback module: used to determine the reward function through the reinforcement learning algorithm according to the test results, and adjust the test priority of each module of the object to be tested and the test weight of the test case according to the reward function.

[0042] A third aspect of the embodiment of the present application further provides a software testing device, the device comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other through the communication bus;

[0043] The memory is used to store at least a program, and the program enables the processor to execute the operations of the software testing method provided in the above embodiment.

[0044] According to a fourth aspect of the embodiments of the present application, a computer-readable storage medium is provided, in which at least one program is stored. When the program is run on a software testing device, the intelligent software testing device performs the operations of the intelligent software testing method provided in any of the above embodiments.

[0045] As can be seen from the above, the embodiment of the present application identifies the test priority of each module of the object to be tested, matches multiple test cases with each module according to the test priority, assigns different test weights to each test case, and then crosses and mutates the execution path of the test case according to the test weight and test priority through a genetic algorithm to determine the optimal test case execution path. In the testing process, a reinforcement learning algorithm is used to provide real-time test defect feedback based on the test results. According to the defect feedback results, the test priority of each module of the object to be tested and the test weight of the test case are adjusted through a reward function, so that the software testing platform can adaptively adjust the test execution path according to the test results during the testing process, thereby greatly improving the testing efficiency and the defect detection rate.

[0046] The above description is only an overview of the technical solution of the embodiment of the present application. In order to more clearly understand the technical means of the embodiment of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the embodiment of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The accompanying drawings are only used to illustrate the embodiments and are not to be considered as limiting the present application. In addition, the same reference symbols are used to represent the same components throughout the accompanying drawings. In the accompanying drawings:

[0048] Figure 1 A schematic diagram of the structure of a software testing system provided in an embodiment of the present application;

[0049] Figure 2 A schematic diagram of the structure of a software testing platform provided in an embodiment of the present application;

[0050] Figure 3 A schematic diagram of the structure of a software testing algorithm platform provided in an embodiment of the present application;

[0051] Figure 4 A schematic diagram of a data processing platform structure provided in an embodiment of the present application;

[0052] Figure 5 A schematic diagram of the structure of a test management platform provided in an embodiment of the present application;

[0053] Figure 6 A schematic diagram of the structure of a monitoring platform provided in an embodiment of the present application;

[0054] Figure 7 A schematic diagram of the structure of a system management platform provided in an embodiment of the present application;

[0055] Figure 8 A flowchart of a software testing method provided in an embodiment of the present application;

[0056] Fig. 9 A flowchart of another software testing method provided in an embodiment of the present application;

[0057] Fig.10 A schematic diagram of a defect prediction method flow chart provided in an embodiment of the present application;

[0058] Fig.11 A schematic diagram of a genetic algorithm flow chart provided in an embodiment of the present application;

[0059] Fig.12 A schematic diagram of the structure of a software testing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0060] The exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein.

[0061] Software testing plays a very important role in the software development process. Software testing at different stages of software development can discover and fix potential problems and errors, thereby improving the overall quality of the software and user satisfaction. Software testing can help developers discover and solve potential problems in advance, thereby avoiding major vulnerabilities found after release and reducing the risk of software development. By conducting sufficient software testing, it can ensure that all functions of the software run normally and improve the reliability of the software. Software testing is not only the responsibility of testers, but also the responsibility of developers. By promoting collaboration between developers and testers, the quality of software and development efficiency can be improved.

[0062] Existing software testing platforms usually rely on manually written test cases and scripts, with a low degree of automation. Even if there are some automated functions, they often only cover basic test scenarios and cannot cope with complex test requirements, resulting in low test efficiency. At the same time, many existing testing tools lack intelligent processing capabilities and cannot use technologies such as big data, machine learning or natural language processing to conduct in-depth analysis and automated processing of test data, which results in the analysis and optimization of test results still relying on manual judgment, which is prone to misjudgment and omissions.

[0063] By applying artificial intelligence (AI) technology to the software testing process, the intelligent testing function can be realized, which can greatly improve the degree of automation of software testing. It can also conduct in-depth analysis of test data and improve the intelligence level of software testing.

[0064] However, the inventors of the present application discovered in their research that the existing methods of applying artificial intelligence technology to software testing usually use static test strategy management methods, which make it difficult to dynamically adjust the test path according to the real-time changing environment or system status. The test system lacks adaptive capabilities and is unable to respond to complex or sudden system changes in a timely manner, which affects the reliability and effectiveness of the test results, makes it impossible to effectively optimize the test strategy, and lacks a process for adaptively and dynamically adjusting the test strategy during the test execution process. It is impossible to dynamically adjust the test path according to the test results, thereby improving the test effect, as in the case of manually executing test cases.

[0065] In view of this, the embodiment of the present application proposes a software testing management system, a software testing platform, a software testing method and a software testing device. In order to solve the problem that the test strategy cannot be adaptively and dynamically adjusted in the prior art, resulting in unsatisfactory test results, the embodiment of the present application identifies the test priority of each module of the object to be tested, matches multiple test cases with each module according to the test priority, assigns different test weights to each test case, and then crosses and mutates the execution path of the test case according to the test weight and test priority through a genetic algorithm to determine the optimal test case execution path, and in the test process, the test defect feedback is performed in real time according to the test result through a reinforcement learning algorithm, and the test priority of each module of the object to be tested and the test weight of the test case are adjusted through a reward function according to the defect feedback result, so that the software testing platform can adaptively adjust the test execution path according to the test result during the test process, greatly improving the efficiency of the test and also improving the defect detection rate.

[0066] like Figure 1 As shown, a structural diagram of a test system proposed in an embodiment of the present application is shown, and the test system includes a software testing platform 100, an algorithm platform 200, a data processing platform 300, a test management platform 400, a monitoring platform 500 and a system management platform 600.

[0067] The software testing platform 100 is mainly used to test software, predict and analyze defects, and analyze test results by using machine learning and natural language processing technologies. Figure 2 As shown, the software testing platform 100 includes a data acquisition module 101, a data optimization module 102, a defect prediction module 103, a test execution module 104, a defect feedback module 105 and a result analysis module 106. The software testing platform 100 is one of the core modules of the test system, which is mainly used to execute the software testing method proposed in the embodiment of the present application to solve the problem in the prior art that the test strategy cannot be adaptively and dynamically adjusted during the test process.

[0068] The data acquisition module 101 is mainly used to obtain multiple groups of test cases and information of the object to be tested, and identify the test priority of each module of the object to be tested according to the information of the object to be tested, wherein the test priority represents the test order of each module. For modules with higher defect risks, higher software complexity or newly added modules, a higher test priority can be set to give priority to testing these modules.

[0069] The data optimization module 102 is mainly used to match the associations of multiple groups of test cases with each module, determine the degree of association between each test case and each module, and generate test weights corresponding to the test cases according to the test priorities and the degrees of association of each module, and perform crossover and mutation on the execution paths of the test cases through a genetic algorithm according to the test weights to determine the optimal execution path of each test case.

[0070] The test execution module 104 is mainly used to use machine learning and natural language processing technology to execute the optimal test case according to the execution path and obtain the test result corresponding to the test case.

[0071] The defect feedback module 105 is mainly used to determine the reward function through the reinforcement learning algorithm according to the test results, and adjust the test priority of each module of the object to be tested and the test weight of the test case according to the reward function, and feed back the analysis and processing results of the test results to the developer.

[0072] The defect prediction module 103 is used to obtain historical test log data and train a defect prediction model based on the historical test log data; it also predicts defects on the object to be tested based on the trained defect prediction model, and adjusts the test weight of the test case based on the defect prediction result.

[0073] The result analysis module 106 is mainly used to analyze the test results using machine learning algorithms, identify defect patterns and defect classifications, and automatically generate test report summaries using integrated natural language generation (NLG) technology. Alternatively, key information (such as defect classification and impact range) is extracted from the test results, and report summaries and suggestions are automatically written through template matching and generation models (such as GPT-2). Through intelligent defect association and analysis capabilities, complex defects are deeply tracked and managed, the efficiency of the defect repair process is improved, and duplicate or unrepaired defects are avoided.

[0074] The software testing platform 100 is mainly used to execute the process and steps of the software testing method, which will be described in detail in subsequent embodiments.

[0075] The algorithm platform 200 is mainly used to test and analyze software by setting test requirements, programming languages ​​and targets. The algorithm platform 200 cooperates with the software testing platform 100 to complete the software testing work. Figure 3 As shown, the algorithm platform 200 includes a test requirement specification model module 201, a programming language model module 202, a target model module 203, a test case model module 204 and a test analysis model module 205. The test requirement specification model module 201 is mainly used to collect historical test data to obtain the test requirement specification manual so that the test process meets the requirements of the test specification. The programming language model module 202 is mainly used to determine the range of programming languages ​​that need to be modeled, collect programming specifications and related programming examples for each programming language, and parse them one by one according to the items of the programming specification to obtain the semantic set of the programming specification. The target model module 203 is mainly used to build a semantic model matching programming language model based on the target system user manual. The test case model module 204 is mainly used to store different types of software test cases. The test analysis model module 205 is used to record the test result feedback of each test case, analyze the failed test cases, record the cause of the problem in combination with the keywords in the programming language model, give the reason for failure according to the representation meaning corresponding to the keywords, and give a risk description and report; at the same time, based on the programming language model, give a risk elimination solution for the risk code segment with similar code passing the test, and for the passed test cases, further analyze whether the programming specifications meet the standards in combination with the programming language model, and give a programming specification analysis report.

[0076] The data processing platform 300 is mainly used for data cleaning, data classification, building semantic sets, data semantic association and analysis based on test data. Figure 4The method comprises a data cleaning module 301, a data classification module 302, a semantic set construction module 303 and a data semantic association module 304. The data cleaning module 301 is mainly used to clean the acquired historical test data and remove invalid data, such as empty files, untested data and meaningless character data. The data classification module 302 is mainly used to classify the input test data into languages, classify and classify text, character expressions, character codes and digital expressions, and construct knowledge databases respectively. The semantic set construction module 303 is mainly used to construct text word meaning semantic sets for the knowledge database of text, character expression semantic sets for character expressions, code semantic sets for character codes and digital expression semantic sets for digital expressions, and clean and segment the semantic sets. The data semantic association module 304 is mainly used to associate the semantic set databases of test purposes, test requirements, test principles, codes to be tested and test cases, and form a test system association semantic chain library from aspects such as test purposes, test requirements, test principles, codes to be tested and test cases.

[0077] The test management platform 400 is used to manage the project and resource testing methods and processes of the software to be tested, such as Figure 5 As shown, the test management platform 400 includes a test project management module 401, a test resource management module 402 and an AI training management module 403, wherein the test project management module 401 is used to manage test startup, test requirements, test cases, test defects, and test scenarios, and perform automated test execution and monitoring, and perform statistics on test results. The test resource management module 402 is mainly used to manage the mobile devices used by the execution machine and the test equipment. The AI ​​training management module 403 is mainly used for visual algorithm library management, text algorithm library management, image recognition management, text recognition management, and cache management.

[0078] The monitoring platform 500 is mainly used to monitor the software testing process in real time. Figure 6 As shown, the monitoring platform 500 includes a performance monitoring module 501 and an automatic monitoring module 502. The performance monitoring module 501 is used to perform reliability testing and performance testing on the software. The automatic monitoring module 502 is used to perform interface monitoring, UI monitoring and APP monitoring on the software.

[0079] The system management platform 600 is mainly used to configure and set permissions for the test background. Figure 7The system management platform 600 includes a system configuration module 601 and a basic data module 602. The system configuration module 601 is mainly used to set the module configuration, object configuration, authority configuration, process configuration and configuration options. The basic data module 602 is mainly used to set the basic data of user role data, menu data, test items, application systems and localized resources.

[0080] The above-mentioned software testing management system is the basis for the operation of the software testing method and equipment proposed in the embodiment of the present application, and the software testing platform and software testing equipment proposed in the embodiment of the present application are part of the test management system. Based on the software testing management system, the embodiment of the present application proposes a software testing method, which uses the software testing platform 100 in the software testing management system as the execution subject. During the execution process, the software testing platform 100 interacts with the algorithm platform 200, the data processing platform 300, the test management platform 400, the monitoring platform 500 and the system management platform 600 to complete the execution of the solution.

[0081] Specifically, Figure 8 As shown, the software testing method includes the following steps:

[0082] S110: Acquire multiple groups of test cases and information of objects to be tested;

[0083] The information of the object to be tested includes the specification, detailed design, user manual and other software-related documents of the software to be tested. The basic situation of the software can be understood through the information of the object to be tested.

[0084] The test case is a test case pre-generated according to the object to be tested. In the embodiment of the present application, the test case can be automatically generated by using machine learning and natural language processing technology. Fig. 9 As shown, before step S110, the automatic generation of test cases can be performed through the following steps, specifically including:

[0085] S101: Obtaining the requirement document information of the object to be tested;

[0086] The information of the object to be tested includes requirement document information. The requirement document information is obtained by analyzing and extracting the information of the object to be tested. The requirement document information generally includes project background, business requirements, user requirements, functional requirements, data model, system architecture, test requirements and other information.

[0087] S102: performing word segmentation and dependency syntax analysis on the requirement document information to extract key semantic features;

[0088] In an embodiment of the present application, a natural language processing (NLP) model, such as a pre-trained language model such as BERT or GPT, can be used to implement semantic understanding of the requirement document through the Transformer architecture, perform word segmentation and dependency syntactic analysis, and use the encoder layer of BERT to extract key semantic features in the requirement document. This can achieve analysis and extraction of information on the object to be tested, and then extract key semantic feature information of the object to be tested, wherein the key semantic feature information is key technical points or functional points that can reflect the characteristics and functions of the object to be tested, such as module name, key functional points, key codes, etc. The object to be tested can be accurately understood by analyzing and extracting these key technical points or functional points, thereby accurately generating corresponding test cases.

[0089] S103: Analyze the key semantic features through a logistic regression model to generate multiple groups of test cases.

[0090] Logistic regression models are usually used for binary classification problems. They map the output of a linear model to between 0 and 1 through a sigmoid function to predict the probability of an event. In the field of software testing, logistic regression models can be used for semantic analysis to automatically generate test cases. Logistic regression models can be used in conjunction with other machine learning techniques, such as decision trees and random forests, to improve the accuracy and coverage of test case generation. In addition, parameter estimation of logistic regression models is usually achieved through maximum likelihood estimation (MLE), which involves optimization algorithms, such as gradient descent, to find the best fitting parameters.

[0091] In the process of generating test cases through the logistic regression model, NLP technology can be used. The logistic regression model analyzes the requirements document and extracts key information, such as functional points, input conditions, expected outputs, etc. This information will be used as the input features of the model. The extracted requirements features are processed, including word segmentation, part-of-speech tagging, named entity recognition, etc., to construct a data format suitable for the input of the logistic regression model. At the same time, the tokenized requirements document is used as training data to train the logistic regression model. The model learns how to predict the success or failure of the test case based on the information in the requirements document. The new requirements document is input into the trained logistic regression model, and the model will output the probability of success or failure of the test case. Based on these probabilities, test cases can be generated, including test steps and expected results.

[0092] Furthermore, the logistic regression model can also be used to optimize test cases. The output of the logistic regression model can be combined with other NLP techniques such as text summarization and sentiment analysis to further optimize test cases and ensure that they cover all important test scenarios. The generated test cases are executed and the accuracy of the logistic regression model is evaluated based on the test results. The model parameters are adjusted based on the feedback to improve the quality and efficiency of test case generation.

[0093] The above embodiment automatically generates test cases through a logistic regression algorithm, and can adjust model parameters according to feedback, thereby improving the quality and efficiency of test case generation. After step S110, it further includes:

[0094] S120: Identifying the test priority of each module of the object to be tested according to the information of the object to be tested;

[0095] In order to improve the test efficiency, in an embodiment of the present application, different test cases are matched to modules with high test priority of the object to be tested, and different test weights are assigned to different test cases.

[0096] The system needs to identify the test priority of each module of the software to be tested, such as defect risk level, software complexity, etc. Therefore, it is necessary to analyze the object to be tested and determine the test priority of each module of the object to be tested. The identification of defect risk level can be achieved through a preset risk assessment model or user customization.

[0097] When automatically identifying the risk level of each module through the risk assessment model, the risk level of each module can be automatically identified through the following dimensions, thereby determining the test priority of each module, including but not limited to:

[0098] Code complexity: The more complex the code, the more likely it is to have defects, the higher the risk level, and the higher the testing priority;

[0099] Historical defect rate: Modules that have had multiple defects in the past also have a higher risk;

[0100] Code change frequency: Modules that have been frequently modified recently are most likely to introduce new defects, have a higher risk level, and have a higher testing priority;

[0101] Business criticality: For modules with business critical functions, problems will have a significant impact on the system. Even if the code complexity is not high, they need to be tested first. Therefore, they have a higher risk level and a higher testing priority.

[0102] User-defined module: refers to a high-risk module specified by the user.

[0103] Different risk weights and scores can be assigned to each of the above assessment dimensions, representing their respective test priority scores. This information is included in the information of the object to be tested, and different test priorities are assigned to different modules according to the risk assessment results through the risk assessment model algorithm.

[0104] Of course, in addition to determining the test priority of each module of the object to be tested through the risk assessment model algorithm in the above embodiment, the user can also customize the test priority of each module of the object to be tested. The test system provides an input interface to the user. When the user enters the test priority for a specific module, the software testing platform receives the test priority of each module input by the user in response to the user's operation on the modules of the object to be tested.

[0105] S130: performing correlation matching between the plurality of groups of the test cases and the modules, determining the correlation between the test cases and the modules, and generating a test weight corresponding to each test case according to the test priority and the correlation of each module;

[0106] After generating test priorities for each module of the object to be tested, the software testing platform identifies modules with high test priorities based on the test priorities. It is necessary to map and match the generated test cases with modules with high test priorities, and give priority to testing modules with high test priorities. Each test case may cover different functional modules, and it is necessary to determine their association with modules with high test priorities. The system determines whether each test case covers modules with high test priorities based on the code paths involved. If a test case involves a high-risk module, the test case needs to be assigned a higher test weight.

[0107] When assigning test weights to test cases, test weights can be assigned to each test case based on the test priority of each module and the correlation between the module and the test case. For example, a certain threshold can be set for the correlation. Only when the correlation between the module of the object to be tested and the test case is higher than the threshold, the impact of the test priority of the module on the weight of the test case will be considered. For modules with low correlation, the impact of the test priority of the module on the weight of the test case can be ignored. The correlation can also be associated with the weight value. For example, the product of the correlation between the module and the test case and the weight value of the module can be used as the weight value of the test case.

[0108] The test weight of each module can be determined by factors such as code complexity, historical defect rate, business criticality, etc. For example:

[0109] For modules with higher code complexity, the test weight is 5, for example: complexity>10;

[0110] For modules with a high historical defect rate, the test weight is 10, for example: the defect rate is greater than 30%;

[0111] Modules with extremely high business criticality have a test weight of 9;

[0112] The test weight of a test case is calculated comprehensively based on the test weights of the modules it covers.

[0113] For example, test case A covers three modules:

[0114] Module 1: Test weight 5

[0115] Module 2: Test weight 7

[0116] Module 3: Test Weight 2

[0117] The test weight of test case A is 5+7+2=14.

[0118] For user-defined weights, if the user specifies a module as a high test priority module, the test weight of the module will be significantly improved. At this time, the test weights of related test cases will also be significantly improved, and the genetic algorithm will give priority to these test cases for execution.

[0119] After different test weights are assigned to each test case, in order to improve the accuracy of the test weights assigned to each test case, in the embodiment of the present application, the defect prediction model can also be used to predict the results of testing using the test case and its test weight information before the formal test, and the test weight of the test case can be adjusted according to the prediction results. Fig.10 As shown, specifically including:

[0120] S131: Obtain historical test log data;

[0121] S132: training a defect prediction model according to the historical test log data;

[0122] The defect prediction model can be trained using the Random Forest and XGBoost algorithms. The defect prediction model is trained by extracting data such as code complexity, modification frequency, and developer history from historical test log data as input features of the model. The model performance is verified using the cross-validation method to select the optimal hyperparameter combination.

[0123] S133: performing defect prediction on the object to be tested according to the trained defect prediction model, and adjusting the test weight of the test case according to the defect prediction result.

[0124] In an embodiment of the present application, the test results of each preliminarily determined test case are predicted through a defect prediction model, and the test weight of the test case is adjusted according to the prediction result, so that the test weight can more accurately reflect the test priority of each module of the object to be tested, thereby improving the accuracy of the test.

[0125] S140: performing crossover and mutation operations on the execution paths of the test cases through a genetic algorithm according to the test weights and the test priorities of the modules, so as to determine the optimal execution path of each of the test cases;

[0126] Genetic Algorithm (GA) is a search heuristic algorithm that simulates the biological evolution process and is widely used to solve optimization and search problems. The basic idea of ​​GA is to learn from the mechanisms of natural selection, inheritance, crossover and mutation in biological evolution and continuously optimize the solution of the problem through an iterative process.

[0127] The basic operation process of genetic algorithm includes initialization, individual evaluation, selection operation, crossover operation, mutation operation and termination condition judgment. Among them, the initialization process is to encode the solution of the problem into chromosomes (or individuals), usually binary strings, real vectors or other forms of encoding, and then randomly select a group of candidate solutions as the initial population, and then evaluate the fitness of each individual in the population through a pre-defined fitness function, that is, their quality as a solution to the problem. According to the fitness of the individual, the better individuals are selected to participate in the generation of the next generation. The purpose of selection is to directly inherit the optimized individuals to the next generation or to generate new individuals through pairing and crossover and then inherit them to the next generation. The selection operation is based on the fitness evaluation of individuals in the population. The crossover operation is to apply the crossover operator to the population, perform crossover operations according to a certain probability, and produce new offspring. The crossover operation simulates the reproduction process of organisms. The mutation operation is to apply the mutation operator to the population, randomly change certain genes of certain individuals with a certain probability, increase the diversity of the population, and prevent the algorithm from converging prematurely. After the selection, crossover, mutation and other operations, the population obtains the next generation of population. When the termination conditions are met, such as when the preset number of iterations, fitness threshold or other termination conditions are reached, the algorithm stops and outputs the current optimal solution.

[0128] In the embodiment of the present application, in the initialization phase, the execution paths of multiple test cases are randomly selected from multiple groups of test cases as the initial population, and the execution path of each test case is an individual in the population. After the execution paths of the test cases are cross-operated and mutated by the fitness function, a new test case combination is generated, the execution paths of the test cases are optimized, and the optimal test case execution path is determined.

[0129] S150: Execute the test case according to the optimal execution path, and obtain the test result corresponding to the test case;

[0130] The test object is tested according to the optimal execution path of the test case determined by the genetic algorithm, and the test results are obtained in real time.

[0131] S160: Determine a reward function through a reinforcement learning algorithm according to the test result, and adjust the test priority of each module of the object to be tested and the test weight of the test case according to the reward function.

[0132] During the testing process, real-time defect prediction is performed on the object to be tested. In an embodiment of the present application, real-time defect prediction is performed through a reinforcement learning algorithm, and the optimal execution path of the test case is dynamically adjusted to preferentially cover modules with high test priority of the object to be tested.

[0133] Reinforcement learning is an important branch of machine learning, which focuses on how to take actions in the environment to maximize a certain cumulative reward. The core of the reinforcement learning algorithm is that the agent learns the optimal strategy through interaction with the environment, that is, the rule for selecting the best action under a given state. In an embodiment of the present application, the test results are analyzed by the reinforcement learning algorithm, such as: the number of defects, the severity of defects, etc. are analyzed, and a reward function is designed according to the number of defects and the severity of defects, and the test priority of each module of the object to be tested and the test weight of the test case are adjusted according to the reward function, so that each test can be tested with the optimal test path, so as to achieve the purpose of dynamically adjusting the combination of test cases and the execution path of test cases according to the test results and the defect situation in the process of testing.

[0134] It can be seen from the above embodiments that by identifying the test priority of each module of the object to be tested, multiple test cases are matched with each module according to the test priority, and different test weights are assigned to each test case. Then, the test priority of each module of the object to be tested and the test weight of the test case are adjusted through genetic algorithm and reinforcement learning algorithm, so that the software testing platform can adaptively adjust the test execution path during the testing process according to the test results, which greatly improves the test efficiency and the defect detection rate.

[0135] Furthermore, in a genetic algorithm, when performing crossover and mutation operations on individuals, the selection of crossover probability and mutation probability will greatly affect the behavior and performance of the genetic algorithm. In the embodiment of the present application, in order to improve the test efficiency, the embodiment of the present application also proposes a method for optimizing the execution path of a test case through a genetic algorithm, for specific reference Fig.11 ,include:

[0136] S141: Randomly select execution paths of multiple test cases from the multiple groups of test cases as an initial population;

[0137] In an embodiment of the present application, execution paths of multiple test cases are randomly selected from all generated test cases to form an initial population, and each individual in the initial population represents an execution path of a test case.

[0138] S142: determining a fitness function according to the historical defect rate of the object to be tested, the test weight of the test case, and the test priority of each module of the object to be tested;

[0139] The fitness function is used to evaluate the quality of each test case execution path. In the embodiment of the present application, the fitness function can be designed according to the following indicators:

[0140] Historical defect rate: If the modules covered by the test cases have had many defects in history and have a high defect rate, the weight of these test cases will be higher;

[0141] Test priority of each module of the object to be tested: Modules with high test priority should have higher fitness, especially user-defined modules with high test priority, which will be given higher weight;

[0142] Test weight of test cases: For test case execution paths that cover modules with high test priority, the test case should have high fitness;

[0143] Execution Time: The shorter the execution time of a test case, the higher the fitness, which helps in executing more test cases in a limited time.

[0144] S143: Determine the fitness corresponding to each individual in the initial population according to the fitness function;

[0145] According to the fitness function determined in the above steps, the fitness corresponding to each individual is determined, that is, the fitness of each test case execution path is evaluated through the fitness function to determine the optimal test case execution path.

[0146] S144: assigning a crossover probability and a mutation probability to each of the individuals according to the fitness;

[0147] In genetic algorithms, the selection of two parameters, crossover probability and mutation probability, will greatly affect the behavior and performance of the genetic algorithm. For different optimization problems, repeated experiments are required to determine the crossover probability and mutation probability.

[0148] The crossover probability refers to the probability of randomly crossing alleles between parents. The larger the crossover probability, the faster the population can produce new individuals. However, a crossover probability that is too large will lead to a greater possibility that the genetic pattern will be destroyed, which will quickly destroy the individual structure that already has high fitness and is not conducive to the intergenerational transmission of excellent individuals. However, if the crossover probability is too small, the search process will become very slow, which is not conducive to the convergence of the algorithm.

[0149] The mutation probability refers to the probability of a parental gene mutation in the process of generating new individuals. When considering how to avoid premature population maturation, a large mutation probability method is usually used. However, for the mutation probability, although a large mutation probability can continuously update the diversity of the population, it will also destroy the existing optimal solution in the population and turn it into a pure random search algorithm, which is not conducive to algorithm convergence and affects algorithm performance. Therefore, it is very important to assign appropriate crossover probability and mutation probability.

[0150] S145: Determine whether the fitness is greater than or equal to a preset first threshold, if yes, execute step S146, if no, execute step S147;

[0151] In order to ensure the characteristics of existing excellent individuals during the crossover and mutation process, in the embodiment of the present application, an adaptive crossover probability and mutation probability allocation method is adopted so that they can be adaptively changed according to fitness. When the fitness is greater than or equal to the preset first threshold, the first crossover probability and the first mutation probability are allocated to the individual; when the fitness is less than the preset first threshold, the second crossover probability and the second mutation probability are allocated to the individual, wherein the first crossover probability is less than the second crossover probability, and the first mutation probability is less than the second mutation probability. For the population individuals with higher fitness, a lower crossover probability and a lower mutation probability are allocated to them, so that the excellent characteristics of the individual can be protected and passed on to the next generation. For the population individuals with lower fitness, a higher crossover probability and a higher mutation probability are allocated to them, so that the individual can be quickly eliminated. Since the adaptive algorithm can not only ensure that good individuals are transmitted between generations, but also timely eliminate individuals that do not adapt to environmental requirements, it can provide the best crossover probability and mutation probability.

[0152] S146: assigning a first crossover probability and a first mutation probability to the individual according to the maximum fitness value in the population and the average fitness value in the population;

[0153] In the embodiment of the present application, preferably, the first crossover probability P c1 and the first mutation probability P m1 , determined by the following adaptive algorithm:

[0154]

[0155] Among them, F max is the maximum fitness value in the population, F avg is the average fitness value of the population, set to the preset first threshold, F is the larger fitness value of the two individuals to be crossed, and F ′ To mutate the fitness value of the individual, k1 and k3 are constants, preferably, k1 is 0.5 and k3 is 0.02.

[0156] S147: allocating a second crossover probability and a second mutation probability to the individual according to a preset constant;

[0157] In the embodiment of the present application, preferably, the second crossover probability P c2 and the second mutation probability P m2 , determined by the following adaptive algorithm:

[0158] P c2 =k2,F <F avg ;

[0159] P m2 =k4,F <F avg ;

[0160] Among them, F avg is the average fitness value of the population, which is set to the preset first threshold, F is the larger fitness value of the two individuals to be crossed, k2 and k4 are constants, preferably, k2 is 0.9, and k4 is 0.09.

[0161] S148: Determine whether the test target is covered. If so, execute step S150. If not, repeat step S142 until the test target is covered.

[0162] S149: Determine whether the preset number of iterations has been reached. If so, execute step S150. If not, repeat step S142 until the preset number of iterations has been reached.

[0163] Steps S148 and S149 judge the preset termination conditions of the genetic algorithm. The preset termination conditions include whether all test targets are covered or the preset number of iterations is reached. If any one of them is met, the execution of the genetic algorithm can be terminated.

[0164] If the preset termination condition is not reached, the genetic algorithm continues to be executed; if all test case execution paths have been covered or the preset number of iterations has been reached, the execution of the genetic algorithm is exited.

[0165] After the crossover and mutation operations are performed on the test cases through the genetic algorithm to determine the optimal execution path of each test case, the process proceeds to step S150 to continue the subsequent steps of executing the test cases according to the optimal execution path of the test cases to test the test object.

[0166] The above-mentioned embodiments of the present application adaptively adjust the crossover probability and mutation probability of individuals in the process of crossover and mutation of the execution path of the test case using a genetic algorithm. For individuals in the population with higher fitness, lower crossover probability and mutation probability are allocated to them, while for individuals in the population with lower fitness, higher crossover probability and mutation probability are allocated to them. This can ensure that good individuals are passed on from generation to generation and that individuals that do not adapt to environmental requirements can be eliminated in a timely manner, thereby greatly improving the test efficiency.

[0167] According to another aspect of the embodiment of the present application, a software testing device is provided. Fig.12 , Fig.12 A schematic diagram of the structure of a software testing device according to an embodiment of the present application is shown. The specific embodiment of the present application does not limit the specific implementation of the software testing device.

[0168] like Fig.12 As shown, the software testing device may include: a processor 702 , a memory 706 , a communication interface 704 and a communication bus 708 .

[0169] The processor 702, the memory 706 and the communication interface 704 communicate with each other via a communication bus 708. The memory 706 is used to store at least one program 710, and the program 710 enables the processor 702 to execute the relevant steps in the above software testing method embodiment.

[0170] Specifically, the program 710 may include program code including computer executable instructions.

[0171] The processor 702 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application. The one or more processors included in the drone direction determination device may be processors of the same type, such as one or more CPUs; or may be processors of different types, such as one or more CPUs and one or more ASICs.

[0172] The memory 706 is used to store the program 710. The memory 706 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0173] The program 710 can be specifically called by the processor 702 to enable the software testing device to execute any operation method provided by the above software testing method embodiment, and the details are not repeated here.

[0174] An embodiment of the present application further provides a computer-readable storage medium, in which executable instructions are stored. When the executable instructions are executed on a software testing device, the software testing device executes the software testing method provided in any of the above embodiments.

[0175] An embodiment of the present application also provides a software testing program, which is used to execute the software testing method provided in the above embodiment.

[0176] The algorithm or display provided here are not inherently related to any specific computer, virtual system or other equipment. Various general systems can also be used together with the teaching based on this. According to the above description, it is obvious to construct the structure required for this type of system. In addition, the present application embodiment is not directed to any specific programming language yet. It should be understood that various programming languages ​​can be utilized to realize the content of the present application described here, and the above description of specific languages ​​is to disclose the best mode of implementation of the present application.

[0177] In the description provided herein, a large number of specific details are described. However, it is understood that the embodiments of the present application can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0178] Similarly, it should be understood that in order to streamline the present application and help understand one or more of the various aspects of the invention, in the above description of exemplary embodiments of the present application, the various features of the embodiments of the present application are sometimes grouped together into a single embodiment, figure, or description thereof.

[0179] Those skilled in the art will appreciate that the modules in the devices in the embodiments can be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments can be combined into one module or unit or component, and they can be divided into multiple submodules or subunits or subassemblies. Except that at least some of such features and / or processes or units are mutually exclusive, all features disclosed in this specification (including the accompanying abstract and drawings) and all processes or units of any method or device disclosed in this manner can be combined in any combination. Unless otherwise explicitly stated, each feature disclosed in this specification (including the accompanying abstract and drawings) can be replaced by an alternative feature providing the same, equivalent or similar purpose.

[0180] It should be noted that the above embodiments illustrate the present application rather than limit the present application, and those skilled in the art may design alternative embodiments without departing from the scope. The steps in the above embodiments, unless otherwise specified, should not be understood as limiting the execution order.

Claims

1. A software testing method, characterized in that: include: Get multiple sets of test cases and information about the objects to be tested; Identify the test priority of each module of the object to be tested according to the information of the object to be tested; Performing correlation matching between the plurality of groups of the test cases and the modules, determining the correlation between the test cases and the modules, and generating a test weight corresponding to the test cases according to the test priority and the correlation of the modules; According to the test weight and the test priority of each module, performing crossover operation and mutation operation on the execution path of the test case by using a genetic algorithm to determine the optimal execution path of each test case; Execute the test case according to the optimal execution path, and obtain the test result corresponding to the test case; A reward function is determined according to the test result through a reinforcement learning algorithm, and the test priority of each module of the object to be tested and the test weight of the test case are adjusted according to the reward function.

2. The software testing method according to claim 1, characterized in that: Before obtaining the multiple groups of test cases, the method further comprises: Obtaining the requirement document information of the object to be tested; Performing word segmentation and dependency syntactic analysis on the requirement document information to extract key semantic features; The key semantic features are analyzed by a logistic regression model to generate multiple groups of test cases.

3. The software testing method according to claim 1, characterized in that: The information of the object to be tested includes the code complexity, historical defect rate, code change frequency and business criticality corresponding to each module; The step of identifying the test priority of each module of the object to be tested according to the information of the object to be tested includes: Through a preset risk assessment model, risk assessment is performed on each module of the object to be tested according to the code complexity, historical defect rate, code change frequency and business criticality; The test priority corresponding to each module is determined according to the risk assessment result.

4. The software testing method according to claim 1, characterized in that: The step of identifying the test priority of each module of the object to be tested according to the information of the object to be tested includes: In response to the operation on each module of the object to be tested, the test priority of each module input by the user is received.

5. The software testing method according to claim 3 or 4, characterized in that: After generating the test weights corresponding to the test cases according to the test priorities of the modules, the method further includes: Get historical test log data; Training a defect prediction model based on the historical test log data; Defect prediction is performed on the object to be tested according to the trained defect prediction model, and the test weight of the test case is adjusted according to the defect prediction result.

6. The software testing method according to claim 5, characterized in that: The step of performing crossover and mutation operations on the execution paths of the test cases by using a genetic algorithm according to the test weights and the test priorities of the modules to determine the optimal execution paths of the test cases includes: Randomly select execution paths of multiple test cases from multiple groups of test cases as an initial population, wherein each execution path of the test case is an individual; Determine a fitness function according to the historical defect rate of the object to be tested, the test weight of the test case and the test priority of each module of the object to be tested; The fitness corresponding to each individual of the initial population is determined according to the fitness function, and a crossover operation and a mutation operation are performed on each individual according to the fitness to generate an optimal execution path for each test case until a preset termination condition is met.

7. The software testing method according to claim 6, characterized in that: The performing crossover operation and mutation operation on the individuals according to the fitness to generate the optimal execution path of each of the test cases includes: A crossover probability and a mutation probability are respectively allocated to each of the individuals according to the fitness, and after performing a crossover operation and a mutation operation on the initial population according to the crossover probability and the mutation probability, an optimal execution path of the test case is generated.

8. The software testing method according to claim 7, characterized in that: The step of allocating a crossover probability and a mutation probability to each individual according to the fitness is specifically as follows: If the fitness of the individual is greater than or equal to a preset first threshold, assigning a first crossover probability and a first mutation probability to the individual according to the maximum fitness value in the population and the average fitness value in the population; If the fitness of the individual is less than a preset first threshold, assigning a second crossover probability and a second mutation probability to the individual according to a preset constant; The first crossover probability is smaller than the second crossover probability, the first mutation probability is smaller than the second mutation probability, and the preset first threshold is the average fitness value of the population.

9. A software testing platform, characterized in that: include: Data acquisition module: used to acquire multiple groups of test cases and information of the object to be tested, and identify the test priority of each module of the object to be tested according to the information of the object to be tested; Data optimization module: performing correlation matching between the plurality of groups of test cases and the modules, determining the correlation between the test cases and the modules, and generating the test weights corresponding to the test cases according to the test priority and the correlation of the modules; and performing crossover and mutation operations on the execution paths of the test cases through a genetic algorithm according to the test weights and the test priorities of the modules, so as to determine the optimal execution path of each test case; Test execution module: used to execute the test case according to the optimal execution path and obtain the test result corresponding to the test case; Defect feedback module: used to determine the reward function through the reinforcement learning algorithm according to the test results, and adjust the test priority of each module of the object to be tested and the test weight of the test case according to the reward function.

10. A software testing device, characterized in that: include: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least a program, and the program enables the processor to perform the operation of the software testing method according to any one of claims 1-8.

11. A computer-readable storage medium, characterized in that: The storage medium stores at least one program, and when the program is executed on the software testing device, the software testing device executes the operation of the software testing method according to any one of claims 1 to 8.

Images