Exploratory test strategy generation method and device, electronic equipment and storage medium

Abstract

The invention discloses an exploratory test strategy generation method and device, electronic equipment and a storage medium. The method comprises the steps that a test suite of a product is acquired from a use case library; judging a dependency relationship between any two test suites by adopting an interface dependency model; and generating an exploratory test strategy by adopting a genetic algorithm according to the dependency relationship. On one hand, exploratory testing is carried out on a system function scene and a fault scene, and on the other hand, the search space of the genetic algorithm is reduced by applying the interface dependence model.

Description

Technical Field

[0001] The present application relates to the technical field of test strategy generation, and in particular to an exploratory test strategy generation method, device, electronic device, and storage medium. Background Art

[0002] How to find as many potential product problems as possible and comprehensively before the product is released is a problem that testers have always wanted to solve. Even if they have a lot of testing experience, testing methods and theoretical methods, unexpected problems will always occur after the product is launched.

[0003] Commonly used exploratory testing methods include full random testing, chaos testing, etc., but there are still shortcomings. Summary of the invention

[0004] The embodiments of the present application provide a method, device, electronic device, and storage medium for generating an exploratory test strategy, so as to optimize the generation of an exploratory test strategy based on an interface dependency model and a genetic algorithm.

[0005] The present application embodiment adopts the following technical solutions:

[0006] In a first aspect, an embodiment of the present application provides a method for generating an exploratory test strategy, wherein the method comprises:

[0007] Get the product's test suite from the test case library;

[0008] Using an interface dependency model to determine the dependency relationship between any two of the test suites; and

[0009] According to the dependency relationship, a genetic algorithm is used to generate an exploratory testing strategy.

[0010] In some embodiments, according to the dependency relationship, a genetic algorithm is used to generate an exploratory testing strategy, including:

[0011] According to the directed graph between the suites, an initialization population is defined;

[0012] The fitness of each contemporary individual is calculated using a pre-created fitness function for the initialized population;

[0013] Determine whether the current number of iterations is greater than the preset number of iterations of the constrained genetic algorithm;

[0014] If not, the fitness value of the contemporary individual is compared with the initial threshold;

[0015] If the fitness value is greater than the initial threshold, the individual is passed on to the next generation;

[0016] If the fitness value is less than the initial threshold, the individual is not directly passed to the next generation.

[0017] In some embodiments, if the fitness value is greater than the initial threshold, it can be inherited to the next generation, including:

[0018] Based on the directed graph relationship between suites, select excellent individuals that meet the conditions for cross processing;

[0019] Select, sort and combine the individuals that meet the conditions after cross processing to form new individuals;

[0020] According to the formation of new individuals, the interface dependency model and the genetic algorithm are re-entered to carry out a new round of genetic iteration.

[0021] In some embodiments, if the fitness value is less than the initial threshold, the individual is not suitable for direct transmission to the next generation, including:

[0022] Based on the failure scenario as a variability factor;

[0023] New individuals are generated by injecting the mutation factors, and the interface dependency model and the genetic algorithm are re-entered according to the formation of the new individuals to carry out a new round of genetic iteration.

[0024] In some embodiments, the using an interface dependency model to determine the dependency relationship between any two of the test suites includes:

[0025] According to the test suite of the product, obtain the message sets Req1, Res1, Req2, Res2 of two test suites 1 and 2;

[0026] Process the message sets of the two suites separately and calculate the field similarity ratio between the suites;

[0027] If the similarity ratio is greater than a probability threshold, the two kits are related, otherwise they are not related.

[0028] In some embodiments, the method further comprises:

[0029] If the two suites are related, the similarity of the message fields of {Req1, Res 2} and {Req2, Res1} are analyzed respectively;

[0030] If the message field similarity of {Req1, Res 2} is greater than {Req2, Res1}, it is determined that the operation order between the two suites is suite 2 to suite 1, otherwise the operation order between the suites is suite 1 to suite 2;

[0031] According to the operation sequence and the correlation between the two suites, a directed graph between the test suites is obtained to characterize the correlation between the suites and the operation sequence.

[0032] In some embodiments, after generating an exploratory testing strategy using a genetic algorithm according to the dependency relationship, the method further includes:

[0033] generating an exploratory testing strategy according to the genetic algorithm to obtain a new test suite;

[0034] Repeating the testing of the new test kit;

[0035] If the repetitiveness of the new test suite is greater than a first preset threshold, transferring the test suite to manual processing;

[0036] If the repetition degree of the new test suite is not greater than a first preset threshold, checking the number of test cases in the test suite;

[0037] If the number of test cases in the test suite is greater than a second preset threshold, switching to manual processing;

[0038] If the number of use cases in the test suite is not greater than a second preset threshold, the test suite is included in the use case set.

[0039] In a second aspect, an embodiment of the present application further provides an exploratory test strategy generating device, wherein the device comprises:

[0040] The acquisition module is used to obtain the test suite of the product from the use case library;

[0041] A first generating module is used to determine the dependency relationship between any two of the test suites by using an interface dependency model; and

[0042] The second generating module is used to generate an exploratory testing strategy using a genetic algorithm according to the dependency relationship.

[0043] In a third aspect, an embodiment of the present application further provides an electronic device, comprising: a processor; and a memory arranged to store computer executable instructions, wherein the executable instructions, when executed, cause the processor to perform the above method.

[0044] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, which stores one or more programs. When the one or more programs are executed by an electronic device including multiple application programs, the electronic device executes the above method.

[0045] At least one of the above technical solutions adopted in the embodiments of the present application can achieve the following beneficial effects: first obtain the product test suite from the use case library, then use the interface dependency model to determine the dependency relationship between any two of the test suites, and finally use the genetic algorithm to generate an exploratory test strategy based on the dependency relationship. In the above method, the genetic algorithm is introduced to perform exploratory testing on system function scenarios and fault scenarios, and the interface dependency model is used to reduce the search space of the genetic algorithm and improve the convergence speed of the genetic algorithm.

[0046] In the above method, while greatly reducing manual intervention, the latest use case set can be automatically obtained from the use case library and the genetic results can be eliminated according to the current problem set. Finally, at the end of the iteration, the newly generated test cases that meet the conditions will be automatically updated into the use case library. In this way, the effect of continuous execution and automatic expansion of the use case library is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0048] Figure 1 This is the overall workflow diagram in the embodiment of this application;

[0049] Figure 2 This is a detailed workflow diagram of the test suite structured processing module in the embodiment of the present application;

[0050] Figure 3 This is a detailed workflow diagram of the interface dependency discrimination model in the embodiment of the present application;

[0051] Figure 4 A schematic diagram of a detailed workflow of the genetic algorithm generating an exploratory testing strategy in an embodiment of the present application;

[0052] Figure 5 This is a detailed workflow diagram of the use case push module in the embodiment of this application;

[0053] Figure 6 A schematic diagram of a process for generating an exploratory test strategy in an embodiment of the present application;

[0054] Figure 7 This is a schematic diagram of the structure of an exploratory test strategy generating device in an embodiment of the present application;

[0055] Figure 8 This is a schematic diagram of the structure of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0057] The fully random method in the related technology will randomly match the existing single interfaces in the system and then test them to achieve the purpose of full coverage of the scene. However, this method has a large search space and high time cost, and most of the random combinations have no actual test scenario significance, which is an invalid test. In addition, this method does not have a self-evolution and incentive feedback mechanism, that is, the result of an exploratory combination cannot directly affect the next choice, resulting in the process being infinitely extended as the input set becomes larger, resulting in a significant reduction in exploration efficiency.

[0058] In addition to the fully random method, a more popular engineering method in related technologies is chaos engineering. As the name implies, the purpose of chaos engineering is to randomly inject system failures or arrange failure scenarios to help users simulate various anomalies that may occur in the real world during development testing, and help users discover potential problems in the system. However, chaos testing engineering pays more attention to the random arrangement and injection of system failures, focusing on abnormal scenarios, and does not detect conventional functional scenarios. This method is only a link in the genetic algorithm to detect mutations in test scenarios. It does not meet the needs of current testers to continuously explore and mine functional test scenarios for complex systems. Moreover, in the test scenario of randomly injecting failures in chaos engineering, it is actually impossible to guarantee whether the occurrence of system problems is caused by the failure of conventional functions or by failures, which invisibly increases the difficulty for problem location personnel.

[0059] In view of the above shortcomings, an exploratory test strategy generation method is provided in the embodiment of the present application, which introduces a genetic algorithm to perform exploratory testing on system function scenarios and fault scenarios, and at the same time uses an interface dependency model to reduce the search space of the genetic algorithm and improve its convergence speed. In other words, the genetic algorithm is selectively used to explore new test strategies and assist the interface dependency model, thereby directionally reducing the problems of the genetic algorithm's inherent slow convergence and wide search range.

[0060] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0061] like Figure 1As shown, it is the overall workflow diagram of the embodiment of the present application, which can generate exploratory test strategies end-to-end, and perform model self-optimization and expansion of the use case library according to certain rules. In the workflow, it mainly relies on four modules: test suite structured processing module, interface dependency model module for determining dependencies between test suites, genetic algorithm module for generating test strategies, and use case push module.

[0062] Preferably, a test suite structured processing module is included, the input of which depends on the product's use case library. The module periodically obtains all test suites of the current product from the use case library and extracts content from each test suite: suite name, use case name included in the suite, and interface messages involved in the suite. The use case library is continuously updated with the evolution of iterations, so the input of the device will also be updated in real time.

[0063] Preferably, the interface dependency model determines the dependency relationship module between test suites. The main function of this module is to determine whether two test suites are related. If they are related, the production-consumption theory is used: the call that generates the data should occur before the call that consumes the data, and the order of operations between test suites is obtained. Finally, a directed graph between suites is formed and stored, which provides a judgment condition for selecting suites for crossover and combination later.

[0064] Preferably, the genetic algorithm generates a test strategy module: the genetic algorithm module is the core module of the invention, in which the test suites are selected, crossed, and mutated by defining the number of iterations, fitness function, selection threshold, correlation between suites, etc., and then an excellent new population is generated, and the existing population with poor effect is eliminated. The above operation is repeated until the end condition is reached.

[0065] Optionally, a use case push module is also included: the quality of the use case set generated after the genetic algorithm exits is evaluated, and according to corresponding indicators, it is determined whether the automatically generated use case can be directly entered into the use case library or needs manual adjustment.

[0066] The present application embodiment provides a method for generating an exploratory test strategy, such as Figure 6 As shown, a flow chart of an exploratory test strategy generation method in an embodiment of the present application is provided, and the method at least includes the following steps S610 to S630:

[0067] Step S610: Acquire a test suite for the product from a use case library.

[0068] When obtaining the test suite, it relies on the use case library of the product, and regularly obtains all the test suites of the current product from the use case library, and extracts content from each test suite, including but not limited to: suite name, use case name included in the suite, and interface messages involved in the suite.

[0069] like Figure 2 As shown, filter out valid test suites from the use case set, and extract information from the test suites in turn. Plan to obtain the suite name, interface names and messages in the suite from the use case library. Store the obtained information in the data structure [{"suite name": xxxx, "suite interface information": [{"interface name": xxx, "interface message": xxx}]}]. Loop until all currently valid test suites in the use case library are parsed.

[0070] Step S620: using an interface dependency model to determine the dependency relationship between any two of the test suites.

[0071] Through the interface dependency model, we first identify whether there are dependencies between existing test suites, and use this dependency as an individual selection reference variable for the subsequent genetic algorithm to generate exploratory test strategies. The purpose is to reduce the search space of the genetic algorithm so that it can quickly find and output test solutions that can discover product problems.

[0072] The interface dependency model is used to determine whether two test suites are related. When determining the dependency, the message sets of the two suites are processed separately, the message set fields of the two suites are deduplicated, and then the two message field sets are compared. If the message field exists in the other message, the same field count is increased by 1. Similarly, if it does not exist, the edit distance algorithm is called to count whether there is a field with an edit distance length less than 2 in the other message set. If so, the same field count is also increased by 1.

[0073] Step S630: Generate an exploratory testing strategy using a genetic algorithm according to the dependency relationship.

[0074] In the genetic algorithm, by defining the number of iterations, fitness function, selection threshold, correlation between suites, etc., the test suites are selected, crossed, and mutated to generate excellent new populations and eliminate existing populations with poor results.

[0075] Furthermore, for the offspring individuals that tend to be stable, mutation factors will be injected into them according to the idea of ​​genetic algorithm to break the steady state and achieve better exploration purposes.

[0076] The above method, by applying genetic algorithms to scenarios generated by exploratory testing strategies, can achieve effective exploration of existing functional testing scenarios and fault testing scenarios, thereby improving product quality.

[0077] The above method will continue to run in an end-to-end fully automated manner, and will continuously incorporate the generated new test suites into the input set. It will also use the executed test results and the latest product problem sets discovered online as the results to reversely incentivize the optimization model.

[0078] Different from the existing exploratory testing methods in related technologies, such as the full random method, which will produce too many invalid combinations, such as randomly combining two interfaces that have no correlation, which has no practical significance in the test scenario, greatly reducing the exploration efficiency. The above method uses a genetic algorithm to purposefully select and cross the test suite, so that it can continuously and selectively arrange and construct new functional test scenarios based on the learning results.

[0079] Different from the related technologies, the existing exploratory testing methods cannot automatically select the good and bad results of the previous exploration, which causes the search time to increase infinitely with the increase of the input set, and prolongs the time to find the problem as soon as possible. Using the above method, all test suites can be divided into dependencies in advance based on the interface dependency model, and the dependency relationship between suites can be found, which reduces the search space of the subsequent genetic algorithm during selection and crossover, thereby reducing the generation of invalid test scenarios, achieving rapid convergence, and discovering product problems as soon as possible.

[0080] Different from related technologies, chaos engineering has randomly arranged and injected fault scenarios to achieve the exploration of fault scenarios, but it mostly ignores the exploration of functional scenarios. For large systems, in-depth exploratory testing of functional scenarios is often more practical and can discover product or code logic problems that cannot be discovered manually. The above method uses the mutation factor injection theory in genetic algorithms to enrich exploratory test scenarios, so that it not only meets the exploratory testing needs of functional scenarios but also meets the exploratory testing needs of fault scenarios.

[0081] In summary, in the relevant technologies, since exploratory testing is not a necessary part of the traditional testing process, exploratory testing does not have a relatively unified means and methods like functional testing and performance testing. At present, the exploratory testing method commonly used in the industry is generally to perform a random combination of interfaces in a full arrangement, so that different interfaces collide with each other, so as to discover product problems. However, this method is actually a random combination without any strategy. When the number of interfaces is large, the effect is poor. Moreover, this mechanism cannot make value judgments on the new individuals generated after the combination, and cannot eliminate the combination individuals with no value and meaning in the early stage of the experiment. There is also a more popular engineering method "chaos testing" that is more popular. However, chaos testing focuses on abnormal scenarios and focuses on random arrangement and injection of faults. However, in the actual test context, in addition to fault scenarios, some deeper functional scenario defects are still difficult to discover. Due to the complex logic or serious coupling of the system, there are many and deep association levels and operation links between systems. Conventional testing means or methods are difficult to ensure that such hidden problems are reported before going online.

[0082] The above method is used to solve the above problems very well. This solution can achieve strategic selection and elimination of individuals through genetic algorithms and interface dependency models, rather than blind random selection, which greatly reduces the generation of invalid test cases. In addition, this solution is compatible with the exploration of system functional scenario testing strategies and fault scenario testing strategies, and has more comprehensive coverage than chaos testing. In addition, the exploration and execution of test cases has an inherent priority order. This mechanism believes that functional scenario problems should be discovered first, and fault exploration should be done based on this scenario after the functions are stable. The clear hierarchy reduces the complexity of the system environment when problems occur, and reduces the difficulty for developers to locate problems.

[0083] In one embodiment of the present application, a genetic algorithm is used to generate an exploratory testing strategy based on the dependency relationship, including: defining an initialization population based on the directed graph between the suites; using a pre-created fitness function to calculate the fitness of each contemporary individual for the initialization population; determining whether the current number of iterations is greater than a pre-set number of iterations of the constrained genetic algorithm; if not, comparing the fitness value of the contemporary individual with an initial threshold; if the fitness value is greater than the initial threshold, the individual is inherited to the next generation; if the fitness value is less than the initial threshold, the individual is not directly passed to the next generation.

[0084] Please refer to Figure 4 First, define the initial population. For example, the existing and effective test suites in the use case library can be used as the initial population. Then create a fitness function. The purpose of the fitness function here is to evaluate the ability of the current test suite to find system problems.

[0085] Optionally, the fitness function is:

[0086] Individual fitness = (N + the total number of problems found in the suite in the last iteration) / (N + the total number of executions of the suite in the last iteration), where N is a constant

[0087] In addition to the above variable definitions, the genetic algorithm is also constrained by the number of iterations. It is assumed here that the number of iterations cannot exceed M, where M is a positive integer. Usually, users can dynamically set the number of iterations based on whether the test time is sufficient.

[0088] First, the individual fitness function is applied to the initial population to calculate the fitness of each contemporary individual, that is, to evaluate the ability of the current test suite to discover system problems.

[0089] Secondly, determine whether the current number of iterations is greater than M. If so, exit the iteration; if not, proceed to the next step.

[0090] Finally, the fitness value of the contemporary individual is compared with the default threshold set by the plan. If the fitness value is greater than the threshold, it means that the individual is an excellent individual and can be inherited to the next generation; if the fitness value is less than the threshold, it means that the individual is not suitable for direct transmission to the next generation.

[0091] The above method uses the test suite as the initial population and combines the genetic algorithm to generate the device and process of the scenario exploratory test strategy. This exploratory test strategy generation method can automatically screen the new use cases generated by the survival of the fittest and determine whether they need to be inherited to the next generation; it can effectively help testers find hidden system problems without consuming too much extra manpower. It helps to improve product quality, which is particularly important for financial products with high system complexity.

[0092] In one embodiment of the present application, if the fitness value is greater than the initial threshold, it can be inherited to the next generation, including: based on the directed graph relationship between the suites, selecting excellent individuals that meet the conditions for crossover processing; selecting, sorting, and combining the individuals that meet the conditions after the crossover processing to form new individuals; based on the formation of new individuals, re-entering the interface dependency model and the genetic algorithm to perform a new round of genetic iteration.

[0093] Please refer to Figure 4 , select excellent individuals that meet the conditions for crossover. The selection principle here is based on the directed graph relationship between the suites. The excellent individuals selected for combination should be related to each other, and their combination order is uniquely determined according to the directed graph relationship. Individuals that meet the conditions are selected, sorted, and combined to form new individuals. After the new individuals are formed, they will re-enter the interface dependency judgment module and the genetic algorithm module for a new round of genetic iteration.

[0094] In one embodiment of the present application, if the fitness value is less than the initial threshold, the individual is not suitable for being directly passed to the next generation, including: based on the fault scenario as a mutation factor; generating a new individual by injecting the mutation factor, and re-entering the interface dependency model and the genetic algorithm based on the formation of the new individual to perform a new round of genetic iteration.

[0095] Please refer to Figure 4 For individuals with values ​​less than the threshold, it is not that they have no test value. The reason for this phenomenon may be that the scenario is a core transaction and its code stability is relatively high. At this time, we will inject mutation factors into the test individuals: that is, failure scenarios, to verify the stability of the function under abnormal conditions. New individuals are generated by injecting mutation factors, and the individuals will re-enter the interface dependency judgment module and the genetic algorithm module for a new round of genetic iteration.

[0096] The problem with conventional genetic algorithms is that the search space is large, and many offspring with no practical value will be generated during the iteration process, resulting in excessively high iteration time costs. The above method combines the test scenario with the message production-consumption theory, innovatively introduces the test suite dependency model, and determines the correlation and operation order between suites, effectively reducing the search space of the genetic algorithm during selection and crossover.

[0097] In one embodiment of the present application, the interface dependency model is used to determine the dependency relationship between any two test suites, including: according to the test suite of the product, obtaining the message sets Req1, Res1, Req2, and Res2 of two test suites 1 and suite 2; processing the message sets of the two suites respectively, and calculating the field similarity ratio between the suites; if the similarity ratio is greater than a probability threshold, the two suites are correlated, otherwise they are not correlated.

[0098] Please refer to Figure 3 , obtain the message sets of two test suites from the test suite structured processing module: Req1, Res1, Req2, Res2, where Req stands for Request and Res stands for Response.

[0099] Then, process the message sets of the two suites separately, remove the duplication of the message set fields of the two suites, and then compare the two message field sets. If the message field exists in the other message, the same field count is increased by 1. If not, call the edit distance algorithm to count whether there is a field with an edit distance length less than 2 in the other message set. If so, the same field count is also increased by 1. Finally, the suite similarity is calculated according to the following calculation formula:

[0100] Suite similarity = (number of identical fields * 2) / (the sum of the total number of fields in the two test suites after deduplication)

[0101] Next, the suite similarity obtained in the previous step is compared with the planned threshold. If the similarity is greater than the threshold, it means that the two suites are related; otherwise, they are not related. The correlation calculation formula is as follows:

[0102]

[0103] In the above method, if the relationship is judged to be related, according to the production-consumption theory, the call that generates the data should occur before the call that consumes the data, and the order of operations between test suites is obtained. Finally, a directed graph between suites is formed and stored, which provides judgment conditions for the subsequent selection of suites for crossover and combination.

[0104] In one embodiment of the present application, the method further includes: if the two suites are correlated, respectively analyzing the message field similarities of {Req1, Res 2} and {Req2, Res1}; if the message field similarity of {Req1, Res 2} is greater than that of {Req2, Res1}, determining that the operation order between the two suites is suite 2 to suite 1, otherwise the operation order between the suites is suite 1 to suite 2; according to the operation order and the correlation between the two suites, obtaining a directed graph between the test suites to characterize the correlation and operation order between the suites.

[0105] In specific implementation, if two suites are related, the similarity of the message fields of {Req1, Res 2} and {Req2, Res1} is analyzed respectively. Here, based on the production-consumption theory, it is believed that the call that produces the field should precede the call that consumes the field. In other words, if the field similarity of {Req1, Res2} is greater than that of {Req2, Res1}, it means that the operation order between the two suites is [suite 2, suite 1], otherwise the operation order between the suites is [suite 1, suite 2]. That is, if the sending field of suite 1 will use a large number of the return fields of suite 2, it means that the logical order of operation is suite 2 first and then suite 1.

[0106] Furthermore, if the two suites are related and the operation sequence between the suites has been determined, a directed graph can be created for the two suites based on the above information and the relationship can be stored. By looping the above steps, a directed graph between test suites will eventually be obtained to indicate the correlation and operation sequence between suites.

[0107] Different from the traditional genetic algorithm method of dealing with mutation factors, the above method can adjust the mutation scenario in combination with the actual test background and test factors. In this invention, the fault scenario is used as a mutation factor, and the mutation factor is not used as the first generation cluster, but acts on the offspring that are about to be eliminated during the iteration process. According to the actual test situation, the eliminated test offspring does not necessarily mean that it has no test value. On the contrary, it may be a core transaction with high stability. Therefore, it is necessary to inject the fault scenario into the offspring at this time to achieve automated fault exploratory testing of the core transaction with high stability.

[0108] In one embodiment of the present application, after using a genetic algorithm to generate an exploratory testing strategy based on the dependency relationship, it also includes: generating an exploratory testing strategy based on the genetic algorithm to obtain a new test suite; performing repeat detection on the new test suite; if the repetition degree of the new test suite is greater than a first preset threshold, transferring the test suite to manual processing; if the repetition degree of the new test suite is not greater than the first preset threshold, checking the number of use cases in the test suite; if the number of use cases in the test suite is greater than a second preset threshold, transferring to manual processing; if the number of use cases in the test suite is not greater than the second preset threshold, including the test suite in the use case set.

[0109] Please refer to Figure 5 When the genetic algorithm meets the exit condition, a batch of excellent offspring generated by the genetic algorithm after several iterations can be obtained as a new test suite. The following formula is used to calculate the repeatability of the test case:

[0110] Use case repetition degree = (maximum number of use case repetitions) / (total number of use cases in the suite - number of repetitions of the use case + 1)

[0111] First, the obtained suite is tested for use case duplication. When the use case duplication in the suite is greater than the specified threshold, it is considered that the test scenario can be simplified, and the test suite will be transferred to manual review.

[0112] Secondly, after the new suite passes the use case duplication screening, the number of use cases in the use case suite will be checked. It is generally believed that the number of interfaces contained in a test suite should not exceed 100. If the suite length is greater than a given threshold, it will enter manual review and it is considered that the suite should be split, otherwise the suite can be directly included in the use case set.

[0113] In addition, this application also provides an end-to-end full-process exploratory test fully automated solution, which can continuously obtain the latest input from the use case set and continuously optimize and adjust the model according to the test results. This greatly reduces the burden on testers and eliminates the need to spend extra manpower and resources to intercept quality issues.

[0114] The present application embodiment also provides an exploratory test strategy generating device 700, such as Figure 7 As shown, a schematic diagram of the structure of an exploratory test strategy generation device in an embodiment of the present application is provided, wherein the exploratory test strategy generation device 700 at least includes: an acquisition module 710, a first generation module 720, and a second generation module 730, wherein:

[0115] In one embodiment of the present application, the acquisition module 710 is specifically used to: acquire a test suite of a product from a use case library.

[0116] When obtaining the test suite, it relies on the use case library of the product, and regularly obtains all the test suites of the current product from the use case library, and extracts content from each test suite, including but not limited to: suite name, use case name included in the suite, and interface messages involved in the suite.

[0117] like Figure 2 As shown, filter out valid test suites from the use case set, and extract information from the test suites in turn. Plan to obtain the suite name, interface names and messages in the suite from the use case library. Store the obtained information in the data structure [{"suite name": xxxx, "suite interface information": [{"interface name": xxx, "interface message": xxx}]}]. Loop until all currently valid test suites in the use case library are parsed.

[0118] In one embodiment of the present application, the first generating module 720 is specifically used to: use an interface dependency model to determine the dependency relationship between any two of the test suites.

[0119] Through the interface dependency model, we first identify whether there are dependencies between existing test suites, and use this dependency as an individual selection reference variable for the subsequent genetic algorithm to generate exploratory test strategies. The purpose is to reduce the search space of the genetic algorithm so that it can quickly find and output test solutions that can discover product problems.

[0120] The interface dependency model is used to determine whether two test suites are related. When determining the dependency, the message sets of the two suites are processed separately, the message set fields of the two suites are deduplicated, and then the two message field sets are compared. If the message field exists in the other message, the same field count is increased by 1. Similarly, if it does not exist, the edit distance algorithm is called to count whether there is a field with an edit distance length less than 2 in the other message set. If so, the same field count is also increased by 1.

[0121] In one embodiment of the present application, the second generating module 730 is specifically used to: generate an exploratory testing strategy using a genetic algorithm according to the dependency relationship.

[0122] In the genetic algorithm, by defining the number of iterations, fitness function, selection threshold, correlation between suites, etc., the test suites are selected, crossed, and mutated to generate excellent new populations and eliminate existing populations with poor results.

[0123] Furthermore, for the offspring individuals that tend to be stable, mutation factors will be injected into them according to the idea of ​​genetic algorithm to break the steady state and achieve better exploration purposes.

[0124] In one embodiment of the present application, the second generating module 730 is further used to

[0125] According to the directed graph between the suites, an initialization population is defined; the fitness of each contemporary individual is calculated using a pre-created fitness function for the initialization population; it is determined whether the number of current iterations is greater than the number of iterations of the constrained genetic algorithm that is pre-set; if not, the fitness value of the contemporary individual is compared with an initial threshold; if the fitness value is greater than the initial threshold, the individual is inherited to the next generation; if the fitness value is less than the initial threshold, the individual is not directly passed on to the next generation.

[0126] In one embodiment of the present application, the second generating module 730 is further used to

[0127] Based on the directed graph relationship between the suites, excellent individuals that meet the conditions are selected for cross processing; the individuals that meet the conditions after the cross processing are selected, sorted, and combined to form new individuals; based on the formation of the new individuals, the interface dependency model and the genetic algorithm are re-entered to perform a new round of genetic iteration.

[0128] In one embodiment of the present application, the second generating module 730 is further used to

[0129] Based on the fault scenario as a mutation factor; generating a new individual by injecting the mutation factor, re-entering the interface dependency model and the genetic algorithm according to the formed new individual, and performing a new round of genetic iteration.

[0130] In one embodiment of the present application, the first generating module 720 is further used to

[0131] According to the test suite of the product, obtain the message sets Req1, Res1, Req2, Res2 of two test suites 1 and suite 2; process the message sets of the two suites respectively, and calculate the field similarity ratio between the suites; if the similarity ratio is greater than the probability threshold, the two suites are related, otherwise they are not related.

[0132] In one embodiment of the present application, the first generating module 720 is further used to

[0133] If the two suites are correlated, the message field similarities of {Req1, Res 2} and {Req2, Res1} are analyzed respectively; if the message field similarity of {Req1, Res 2} is greater than that of {Req2, Res1}, it is determined that the operation sequence between the two suites is suite 2 to suite 1, otherwise the operation sequence between the suites is suite 1 to suite 2; according to the operation sequence and the correlation between the two suites, a directed graph between the test suites is obtained to characterize the correlation and operation sequence between the suites.

[0134] In one embodiment of the present application, the first generating module 720 is further used to

[0135] Generate an exploratory test strategy based on the genetic algorithm to obtain a new test suite; perform repeat detection on the new test suite; if the repeatability of the new test suite is greater than a first preset threshold, transfer the test suite to manual processing; if the repeatability of the new test suite is not greater than the first preset threshold, check the number of use cases in the test suite; if the number of use cases in the test suite is greater than a second preset threshold, transfer to manual processing; if the number of use cases in the test suite is not greater than the second preset threshold, include the test suite in the use case set.

[0136] It can be understood that the above-mentioned exploratory test strategy generation device can implement the various steps of the exploratory test strategy generation method provided in the aforementioned embodiment. The relevant explanations about the exploratory test strategy generation method are applicable to the exploratory test strategy generation device and will not be repeated here.

[0137] Figure 8 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. Figure 8 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. The memory may include a memory, such as a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. Of course, the electronic device may also include hardware required for other services.

[0138] The processor, network interface and memory can be interconnected through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 8 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0139] The memory is used to store the program. Specifically, the program may include a program code, and the program code includes a computer operation instruction. The memory may include a memory and a non-volatile memory, and provides instructions and data to the processor.

[0140] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming an exploratory test strategy generation device at the logical level. The processor executes the program stored in the memory and is specifically used to perform the following operations:

[0141] Get the product's test suite from the test case library;

[0142] Using an interface dependency model to determine the dependency relationship between any two of the test suites; and

[0143] According to the dependency relationship, a genetic algorithm is used to generate an exploratory testing strategy.

[0144] The above application Figure 1 The method performed by the exploratory test strategy generation device disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0145] The electronic device may also perform Figure 1 The method for executing the exploratory test strategy generation device in Figure 1 The functions of the illustrated embodiment will not be described in detail in the embodiments of the present application.

[0146] The present application also provides a computer-readable storage medium, which stores one or more programs, wherein the one or more programs include instructions, which, when executed by an electronic device including multiple application programs, enable the electronic device to execute Figure 1 The method performed by the exploratory test strategy generation device in the illustrated embodiment is specifically used to perform:

[0147] Get the product's test suite from the test case library;

[0148] Using an interface dependency model to determine the dependency relationship between any two of the test suites; and

[0149] According to the dependency relationship, a genetic algorithm is used to generate an exploratory testing strategy.

[0150] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0151] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0152] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0153] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0154] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0155] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0156] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0157] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0158] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0159] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A method for generating an exploratory testing strategy, wherein: The method comprises: Get the product's test suite from the test case library; Using an interface dependency model to determine the dependency relationship between any two of the test suites; and According to the dependency relationship, a genetic algorithm is used to generate an exploratory testing strategy.

2. The method of claim 1, wherein: According to the dependency relationship, a genetic algorithm is used to generate an exploratory testing strategy, including: According to the directed graph between the suites, an initialization population is defined; The fitness of each contemporary individual is calculated using a pre-created fitness function for the initialized population; Determine whether the current number of iterations is greater than the preset number of iterations of the constrained genetic algorithm; If not, the fitness value of the contemporary individual is compared with the initial threshold; If the fitness value is greater than the initial threshold, the individual is passed on to the next generation; If the fitness value is less than the initial threshold, the individual is not directly passed to the next generation.

3. The method of claim 2, wherein: If the fitness value is greater than the initial threshold, it can be inherited to the next generation, including: Based on the directed graph relationship between suites, select excellent individuals that meet the conditions for cross processing; Select, sort and combine the individuals that meet the conditions after cross processing to form new individuals; According to the formation of new individuals, the interface dependency model and the genetic algorithm are re-entered to carry out a new round of genetic iteration.

4. The method of claim 2, wherein: If the fitness value is less than the initial threshold, the individual is not suitable for direct transmission to the next generation, including: Based on the failure scenario as a variability factor; New individuals are generated by injecting the mutation factors, and the interface dependency model and the genetic algorithm are re-entered according to the formation of the new individuals to carry out a new round of genetic iteration.

5. The method of claim 1, wherein: The using the interface dependency model to determine the dependency relationship between any two of the test suites includes: According to the test suite of the product, obtain the message sets Req1, Res1, Req2, Res2 of two test suites 1 and 2; Process the message sets of the two suites separately and calculate the field similarity ratio between the suites; If the similarity ratio is greater than a probability threshold, the two kits are related, otherwise they are not related.

6. The method of claim 5, wherein: The method further comprises: If the two suites are related, the similarity of the message fields of {Req1, Res 2} and {Req2, Res1} are analyzed respectively; If the message field similarity of {Req1, Res 2} is greater than {Req2, Res1}, it is determined that the operation order between the two suites is suite 2 to suite 1, otherwise the operation order between the suites is suite 1 to suite 2; According to the operation sequence and the correlation between the two suites, a directed graph between the test suites is obtained to characterize the correlation between the suites and the operation sequence.

7. The method of claim 1, wherein: After the exploratory testing strategy is generated by using a genetic algorithm according to the dependency relationship, the following steps are also included: generating an exploratory testing strategy according to the genetic algorithm to obtain a new test suite; Repeating the testing of the new test kit; If the repetitiveness of the new test suite is greater than a first preset threshold, transferring the test suite to manual processing; If the repetition degree of the new test suite is not greater than a first preset threshold, checking the number of test cases in the test suite; If the number of test cases in the test suite is greater than a second preset threshold, switching to manual processing; If the number of use cases in the test suite is not greater than a second preset threshold, the test suite is included in the use case set.

8. An exploratory test strategy generating device, wherein: The device comprises: The acquisition module is used to obtain the test suite of the product from the use case library; A first generating module is used to determine the dependency relationship between any two of the test suites by using an interface dependency model; and The second generating module is used to generate an exploratory testing strategy using a genetic algorithm according to the dependency relationship.

9. An electronic device, comprising: processor; as well as A memory arranged to store computer executable instructions, which when executed cause the processor to perform the method of any one of claims 1 to 7.

10. A computer-readable storage medium storing one or more programs, which, when executed by an electronic device including a plurality of application programs, causes the electronic device to execute any one of the methods of claims 1 to 7.

Images