Test case sequencing method, device and electronic device
By optimizing the test case sorting method and utilizing fitness functions and random walks, the local optimum problem was solved, resulting in more accurate test case sorting and more efficient defect detection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHINA MOBILE (SUZHOU) SOFTWARE TECH CO LTD
- Filing Date
- 2021-12-08
- Publication Date
- 2026-08-04
AI Technical Summary
Existing test case sorting methods are prone to getting stuck in local optima and are difficult to obtain the global optimal solution, which affects the efficiency of defect detection in regression testing.
By obtaining the first and second test case sequence sets corresponding to the test case set, the target sequence of non-dominated solutions is determined using the fitness function, and the sorting is optimized by random walk processing and roulette wheel algorithm, combined with neighborhood relationship, to improve the probability of the global optimal solution.
It improves the accuracy and objectivity of test case sorting and enhances the efficiency of defect detection in regression testing.
Smart Images
Figure CN116257421B_ABST
Abstract
Description
Technical Field
[0001] This application relates to computer technology, and more particularly to a method, apparatus, and electronic device for sorting test cases. Background Technology
[0002] Test Case Prioritization (TCP) is a technique that obtains the optimal execution order of test cases in a test case set based on a specific optimization goal (sorting criteria). It aims to prioritize the execution of test cases with high defect detection capabilities during regression testing, expose problems as early as possible, carry out software repair work, and reduce software costs.
[0003] The key to TCP technology is determining the prioritization criteria for test cases. These criteria directly affect the execution order of test cases, thus impacting the efficiency of defect detection in regression testing. Existing test case prioritization methods are prone to getting stuck in local optima. Summary of the Invention
[0004] This application provides a method, apparatus, electronic device, and computer-readable storage medium for sorting test cases, which can increase the probability of obtaining the global optimal solution and make the sorting of test cases more accurate and objective.
[0005] The technical solution of this application embodiment is implemented as follows:
[0006] This application provides a method for sorting test cases, including:
[0007] Obtain the first test case sequence set corresponding to the test case set used to test the program under test;
[0008] Obtain the fitness function, and based on the fitness function, determine the first target sequence in the first use case sequence set that belongs to the non-dominated solution;
[0009] Obtain the second test case sequence set corresponding to the test case set, and determine the neighborhood corresponding to each test case sequence in the second test case sequence set;
[0010] Based on the first target sequence and the neighborhood corresponding to each use case sequence in the second use case sequence set, a random walk is performed on the second use case sequence set to obtain the processed target use case sequence set.
[0011] Based on the fitness function, the second target sequence belonging to the non-dominated solution is determined in the target use case sequence set;
[0012] The first target sequence and the second target sequence are added to the preferred sequence set, so that the test case sequences in the preferred sequence set are used as the sorting result of the test case set.
[0013] In the above scheme, determining the first target sequence belonging to the non-dominated solution in the first test case sequence set based on the fitness function includes: using the fitness function to determine the fitness value of each test case sequence in the first test case sequence set; wherein the fitness value includes average statement coverage and effective execution time; and selecting the first target sequence belonging to the non-dominated solution from the first test case sequence set based on the fitness value of each test case sequence.
[0014] In the above scheme, determining the neighborhood corresponding to each use case sequence in the second use case sequence set includes: determining the neighborhood corresponding to each use case sequence in the second use case sequence set based on the number of use case sequences in the second use case sequence set and the number of use cases in the test cases set.
[0015] In the above scheme, there are multiple first target sequences; the step of performing a random walk on the second use case sequence set based on the first target sequences and the neighborhood corresponding to each use case sequence in the second use case sequence set includes: using a roulette wheel algorithm to select a first target sequence as a first reference sequence from the multiple first target sequences; determining a second reference sequence in each neighborhood of the second use case sequence set based on the neighborhood corresponding to each use case sequence in the second use case sequence set; and performing a random walk on the second use case sequence set based on the first reference sequence and the second reference sequence in each neighborhood.
[0016] In the above scheme, determining the second reference sequence within each domain of the second use case sequence set based on the neighborhood corresponding to each use case sequence in the second use case sequence set includes: performing the following processing on each use case sequence in the second use case sequence set: determining multiple local use case sequences belonging to the corresponding domain from the second use case sequence set based on the neighborhood corresponding to the first use case sequence; the first use case sequence being any one of the use case sequences in the second use case sequence set; determining the local use case sequences belonging to the non-dominated solution from the multiple local use case sequences based on the fitness function; and determining the second reference sequence within the corresponding domain of the local use case sequences belonging to the non-dominated solution.
[0017] In the above scheme, determining the second reference sequence within the corresponding domain based on the local use case sequence belonging to the non-dominated solution includes: when there is only one local use case sequence belonging to the non-dominated solution, the determined local use case sequence belonging to the non-dominated solution is taken as the second reference sequence within the corresponding domain; when the number of local use case sequences belonging to the non-dominated solution is at least two, one local use case sequence is randomly selected from the at least two local use case sequences belonging to the non-dominated solution as the second reference sequence within the corresponding domain.
[0018] In the above scheme, the method further includes: determining the dominance relationship between the first target sequence and the second target sequence based on the fitness function; obtaining the use case sequences belonging to the dominance solution in the first target sequence and the second target sequence according to the dominance relationship; and removing the use case sequences belonging to the dominance solution from the preferred sequence set.
[0019] In the above scheme, the method further includes: determining the dominance relationship between the first target sequence and the second target sequence based on the fitness function, and updating the first use case sequence set based on the dominance relationship; using the target use case sequence set as a new second use case sequence set, and updating the preferred sequence set based on the updated first use case sequence set and the new second use case sequence set; when the number of updates to the preferred sequence set reaches a threshold, obtaining the preferred sequence set obtained from the last update, and using the use case sequences in the preferred sequence set obtained from the last update as the sorting result after sorting the test case set.
[0020] This application provides a test case sorting device, comprising:
[0021] The first acquisition module is used to acquire the first test case sequence set corresponding to the test case set used to test the program under test;
[0022] The second acquisition module is used to acquire a fitness function and determine, based on the fitness function, a first target sequence belonging to a non-dominated solution in the first use case sequence set;
[0023] The third obtaining module is used to obtain the second test case sequence set corresponding to the test case set, and to determine the neighborhood corresponding to each test case sequence in the second test case sequence set;
[0024] The random walk processing module is used to perform random walk processing on the second use case sequence set based on the neighborhood corresponding to each use case sequence in the first target sequence and the second use case sequence set to obtain the processed target use case sequence set.
[0025] The determination module is used to determine, based on the fitness function, a second target sequence belonging to the non-dominated solution in the target use case sequence set;
[0026] An adding module is used to add the first target sequence and the second target sequence to the preferred sequence set, so that the test case sequences in the preferred sequence set are used as the sorting result after sorting the test case set.
[0027] In the above scheme, the second obtaining module is further configured to use the fitness function to determine the fitness value of each use case sequence in the first use case sequence set; wherein the fitness value includes average statement coverage and effective execution time; and based on the fitness value of each use case sequence, to select a first target sequence belonging to a non-dominated solution from the first use case sequence set.
[0028] In the above scheme, the random walk processing module is further used to determine the neighborhood corresponding to each test case sequence in the second test case sequence set based on the number of test case sequences in the second test case sequence set and the number of test cases in the test case set.
[0029] In the above scheme, there are multiple first target sequences; the random walk processing module is further used to select one first target sequence as a first reference sequence from multiple first target sequences using the roulette wheel algorithm; based on the neighborhood corresponding to each use case sequence in the second use case sequence set, determine the second reference sequence in each neighborhood from the second use case sequence set; and perform random walk processing on the second use case sequence set based on the first reference sequence and the second reference sequence in each neighborhood.
[0030] In the above scheme, the random walk processing module is further configured to perform the following processing for each use case sequence in the second use case sequence set: based on the neighborhood corresponding to the first use case sequence, determine multiple local use case sequences belonging to the corresponding domain from the second use case sequence set; the first use case sequence is any one of the use case sequences in the second use case sequence set; based on the fitness function, determine the local use case sequences belonging to the non-dominated solution from the multiple local use case sequences; determine the second reference sequence in the corresponding domain of the local use case sequences belonging to the non-dominated solution.
[0031] In the above scheme, the random walk processing module is further configured to, when there is only one local use case sequence belonging to a non-dominated solution, use the determined local use case sequence belonging to a non-dominated solution as the second reference sequence in the corresponding domain; when the number of local use case sequences belonging to a non-dominated solution is at least two, randomly select one local use case sequence from the at least two local use case sequences belonging to a non-dominated solution as the second reference sequence in the corresponding domain.
[0032] In the above scheme, the test case sorting device further includes: a sorting module, used to determine the dominance relationship between the first target sequence and the second target sequence based on the fitness function, and update the first test case sequence set based on the dominance relationship; take the target test case sequence set as a new second test case sequence set, and update the preferred sequence set based on the updated first test case sequence set and the new second test case sequence set; when the number of updates to the preferred sequence set reaches a threshold, obtain the preferred sequence set obtained from the last update, and use the test case sequences in the preferred sequence set obtained from the last update as the sorting result after sorting the test case set.
[0033] This application provides a test case sorting device, comprising:
[0034] Memory, used to store executable instructions;
[0035] A processor, when executing executable instructions stored in the memory, implements the method provided in the embodiments of this application.
[0036] This application provides a computer-readable storage medium storing executable instructions for inducing a processor to execute and implement the method provided in this application.
[0037] In this embodiment, a fitness function is obtained by acquiring a first set of test case sequences corresponding to the test case set used to test the program under test, and a first target sequence belonging to a non-dominated solution is determined based on the fitness function. A second set of test case sequences corresponding to the test case set is obtained, and the neighborhood corresponding to each test case sequence in the second set is determined. A random walk is performed on the second set of test case sequences based on the first target sequence and the neighborhood corresponding to each test case sequence in the second set to obtain a processed target set of test case sequences. Based on the fitness function, a second target sequence belonging to a non-dominated solution is determined in the target set of test case sequences. The first target sequence and the second target sequence are added to a preferred sequence set, so that the test case sequences in the preferred sequence set are used as the sorting result after sorting the test case set. This increases the probability of obtaining a globally optimal solution and makes the sorting of the test cases more accurate and objective. Attached Figure Description
[0038] Figure 1 This is an optional structural schematic diagram of the electronic device 100 provided in the embodiments of this application;
[0039] Figure 2 This is an optional flowchart illustrating the test case sorting method provided in the embodiments of this application;
[0040] Figure 3 This is an optional detailed flowchart of 202 provided in the embodiments of this application;
[0041] Figure 4 This is an optional detailed flowchart of step 203 in an embodiment of this application;
[0042] Figure 5 This is an optional detailed flowchart of 204 provided in the embodiments of this application;
[0043] Figure 6 This is a flowchart illustrating the steps following step 206 provided in the embodiments of this application;
[0044] Figure 7 This is an optional flowchart illustrating the test case sorting method provided in the embodiments of this application. Detailed Implementation
[0045] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0046] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.
[0047] In the following description, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.
[0048] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.
[0049] This application provides a method, apparatus, electronic device, and computer-readable storage medium for sorting test cases, which can increase the probability of obtaining the global optimal solution and make the sorting of test cases more accurate and objective.
[0050] First, the electronic device used for implementing the sorting method of the above-described test cases, as provided in the embodiments of this application, will be described. See also... Figure 1 , Figure 1 This is an optional structural diagram of the electronic device 100 provided in this application embodiment. In practical applications, the electronic device 100 can be implemented as a terminal or a server. The terminal can be a laptop, tablet, desktop computer, smartphone, dedicated messaging device, portable gaming device, smart speaker, smartwatch, etc., but is not limited to these. The server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery network (CDN) services, and big data and artificial intelligence platforms. Figure 1 The illustrated electronic device 100 includes at least one processor 101, a memory 105, at least one network interface 102, and a user interface 103. The various components of the electronic device 100 are coupled together via a bus system 104. It is understood that the bus system 104 is used to implement communication between these components. In addition to a data bus, the bus system 104 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 1 The general labeled all buses as Bus System 104.
[0051] The processor 101 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.
[0052] User interface 103 includes one or more output devices 1031 that enable the presentation of media content, including one or more speakers and / or one or more visual displays. User interface 103 also includes one or more input devices 1032, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.
[0053] The memory 105 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The memory 105 may optionally include one or more storage devices physically located away from the processor 101.
[0054] The memory 105 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 105 described in this application embodiment is intended to include any suitable type of memory.
[0055] In some embodiments, the memory 105 can store data to support various operations. Examples of such data include programs, modules, and data structures, or subsets or supersets thereof. In this embodiment, the memory 105 stores an operating system 1051, a network communication module 1052, a presentation module 1053, an input processing module 1054, and a test case sorting device 1055. Specifically,
[0056] The operating system 1051 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, and driver layer, for implementing various basic business functions and handling hardware-based tasks.
[0057] The network communication module 1052 is used to reach other computing devices via one or more (wired or wireless) network interfaces 102, such as Bluetooth, WiFi, and Universal Serial Bus (USB).
[0058] Presentation module 1053 is configured to enable the presentation of information (e.g., a user interface for operating peripheral devices and displaying content and information) via one or more output devices 1031 (e.g., a display screen, a speaker, etc.) associated with user interface 103;
[0059] The input processing module 1054 is used to detect and translate one or more user inputs or interactions from one or more input devices 1032.
[0060] In some embodiments, the test case sorting device provided in this application can be implemented in software. Figure 1 A test case sorting device 1055 stored in memory 105 is shown. This device can be software in the form of programs and plug-ins, and includes the following software modules: a first determination module 10551, a second determination module 10552, a third determination module 10553, and a recommendation module 10554. These modules are logically sequential and can therefore be arbitrarily combined or further split according to their implemented functions. The functions of each module will be described below.
[0061] In other embodiments, the test case sorting device provided in this application can be implemented in hardware. As an example, the test case sorting device provided in this application can be a processor in the form of a hardware decoding processor, which is programmed to execute the test case sorting method provided in this application. For example, the processor in the form of a hardware decoding processor can be one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.
[0062] The following section will describe the test case sorting method provided in this application embodiment, using exemplary applications and implementations of the terminal provided in the embodiments of this application.
[0063] See Figure 2 , Figure 2 This is an optional flowchart illustrating the test case sorting method provided in the embodiments of this application, which will be combined with... Figure 2 The steps shown are explained.
[0064] Step 201: Obtain the first test case sequence set corresponding to the test case set used to test the program under test;
[0065] Step 202: Obtain the fitness function, and determine the first target sequence belonging to the non-dominated solution in the first use case sequence set based on the fitness function;
[0066] Step 203: Obtain the second test case sequence set corresponding to the test case set, and determine the neighborhood corresponding to each test case sequence in the second test case sequence set;
[0067] Step 204: Based on the first target sequence and the neighborhood corresponding to each use case sequence in the second use case sequence set, perform a random walk on the second use case sequence set to obtain the processed target use case sequence set;
[0068] Step 205: Based on the fitness function, determine the second target sequence in the target use case sequence set that belongs to the non-dominated solution;
[0069] Step 206: Add the first target sequence and the second target sequence to the preferred sequence set, so that the test case sequences in the preferred sequence set are used as the sorting result after sorting the test case set.
[0070] It should be noted that the test case set contains multiple test cases, each used to test the program under test. Here, the program under test can be an application software program or a web page program; this embodiment does not specifically limit the type of the program under test. In actual implementation, the terminal obtains a test case set for testing the program under test. Here, for ease of explanation, the test case set is denoted as T = {T1, T2, ..., T...} i ,...,T n}, where T i Let be the i-th test case in T, where i and n are both positive integers greater than or equal to 2. Furthermore, the program under test contains multiple statements; here, the number of statements in the program under test is denoted as M, where M is a positive integer greater than or equal to 2. Let ET = {ET1, ET2, ..., ET...} i ,...,ET n Let} represent the execution time of the test cases in T, where ET i This represents the execution time corresponding to the execution of the i-th test case. In this embodiment, test cases are numbered from 1 to n, and statements are numbered from 1 to M. The test matrix F, composed of test cases and statements, has n rows and M columns. If the i-th test case covers the j-th statement, then F... ij =1, otherwise F ij =0.
[0071] It should also be noted that the first test case sequence set includes multiple test case sequences, for example, a number of test case sequences of size x, where x is a positive integer greater than or equal to 2. Each test case sequence in the first test case sequence set contains all the test cases in the test case set; that is, the test case sequences in the first test case sequence set are sequences formed by all the test cases in the test case set according to a certain order. It should be understood that the order of the test cases corresponding to each pair of test case sequences in the first test case sequence set is different. Furthermore, in real-world scenarios, the number of test cases in the test case set is large, and the number of test case sequences encompassed by permutations and combinations of these test cases is enormous. Therefore, in this embodiment, only a portion of the test case sequences from all permutations and combinations are selected to form the first test case sequence set. For example, if the number of test cases n in the test case set is 4, that is, the test case set T = {T1, T2, T3, T4}, and the number of test case sequences x in the first test case sequence set is 3, then the 3 test case sequences in the first test case sequence set can be T1-T2-T3-T4, T2-T1-T3-T4, and T1-T2-T4-T3.
[0072] In some embodiments, see Figure 3 , Figure 3 This is an optional detailed flowchart of step 202 provided in the embodiments of this application. Step 202 includes:
[0073] Step 301: Using the fitness function, determine the fitness value of each use case sequence in the first use case sequence set; wherein, the fitness value includes average statement coverage and effective execution time;
[0074] Step 302: Based on the fitness value of each use case sequence, select the first target sequence belonging to the non-dominated solution from the first use case sequence set.
[0075] It should be noted that the fitness function provided in the embodiments of this application can be implemented by the following formulas (1) and (2):
[0076]
[0077]
[0078] Among them, TS i This indicates the position in the execution order of the first test case for statement i; EET is the total time consumed to execute the test cases when maximum statement coverage is first achieved. i This represents the time consumed in executing the i-th test case, where N is the set of test cases executed to achieve maximum statement coverage.
[0079] In actual implementation, the terminal uses a fitness function to determine the fitness value of each use case sequence in the first use case sequence set, and then selects the first target sequence belonging to the non-dominated solution from the first use case sequence set based on the fitness value of each use case sequence.
[0080] In some embodiments, see Figure 4 , Figure 4 This is an optional detailed flowchart of step 203 in an embodiment of this application. Step 202 includes:
[0081] Step 401: Obtain the second test case sequence set corresponding to the test case set;
[0082] Step 402: Based on the number of test case sequences in the second test case sequence set and the number of test cases in the test case set, determine the neighborhood corresponding to each test case sequence in the second test case sequence set.
[0083] In practice, test cases are treated as individual ants, and their neighborhoods are constructed. The specific process of constructing the neighborhoods based on the von Neumann topology is as follows:
[0084] (1) Arrange the ant population in m rows and n columns, ensuring that pop = m × n;
[0085] (2) For an individual i, i∈{1,2,3,…,pop}, construct the local topology according to the following rules: the upper neighbor is N i (1) = (in)mod pop, if N i (1) = 0, then N i (1) = pop; left neighbor is N i (2) = i-1, if (in)mod pop = 0, then N i (2) = i-1+n; the right neighbor is N i (3) = i + 1, if i mod n = 0, then N i (3) = i + 1 + n; the next neighbor is N i (4) = (i+n) mod pop, if N i (4) = 0, then N i (4) = pop; the center point is N i (5) = i.
[0086] After the above operations, we can obtain the neighborhood L(i) = {N(1), N(2), N(3), N(4), N(5)} of ant i. Calculate the fitness values of the 5 individuals in the neighborhood, and obtain the non-dominated solutions in step four. If the number of non-dominated solutions is greater than 1, randomly select one as the best individual AntLbest in the neighborhood of ant i. i .
[0087] In some embodiments, the number of the first target sequences is multiple; see [link to relevant documentation]. Figure 5 , Figure 5 This is an optional detailed flowchart of step 204 provided in the embodiment of this application. Step 204 includes:
[0088] Step 501: Using the roulette wheel algorithm, select one first target sequence as the first reference sequence from multiple first target sequences;
[0089] Step 502: Based on the neighborhood corresponding to each use case sequence in the second use case sequence set, determine the second reference sequence in each neighborhood from the second use case sequence set;
[0090] Step 503: Based on the first reference sequence and the second reference sequence in each domain, perform a random walk on the second use case sequence set.
[0091] In practice, after constructing the neighborhood of an individual ant, the boundary of the ant's random walk is updated. In this embodiment, the boundary of ant i's random walk in the current iteration is updated according to formula (3):
[0092]
[0093] Where t is the current iteration number, T is the maximum iteration number, and d t and c t These are the upper and lower bounds of all variables in the current iteration, respectively. and Let be the maximum and minimum values of ant i across all dimensions in the current iteration, respectively; I be the boundary shrinkage factor; and w be a parameter that is dynamically adjusted with the iteration number t. The antlion or the best individual in the neighborhood that the ants are circling around.
[0094] In some embodiments, step 502 may also be implemented as follows: for each use case sequence in the second use case sequence set, perform the following processing: based on the neighborhood corresponding to the first use case sequence, determine multiple local use case sequences belonging to the corresponding domain from the second use case sequence set; the first use case sequence is any one of the use case sequences in the second use case sequence set; based on the fitness function, determine the local use case sequences belonging to the non-dominated solution from the multiple local use case sequences; determine the second reference sequence in the corresponding domain of the local use case sequences belonging to the non-dominated solution.
[0095] In practice, firstly, the random walk distances of the ant around the elite antlion, the antlion selected by roulette wheel selection, and the best neighboring ant in the current iteration are calculated according to formulas (4) and (5), respectively. Then, the ant position is updated according to formula (6):
[0096] X i =[0,cumsum[2r(1)-1],…,cumsum[2r(t)-1],…,cumsum[2r(T)-1]] (4)
[0097]
[0098]
[0099] in, For the antlion chosen by the antlion in generation t, the ant walks randomly. For the t-th generation, the ants randomly roam around the elite antlion. For the random walk of the ant around the optimal neighbor ant in generation t, a i Let b be the minimum step size for ant i to perform a random walk. iLet be the maximum step size of ant i's random walk. In practice, to keep the ant's random walk within the search space, its position needs to be normalized.
[0100] In some embodiments, determining the second reference sequence within the corresponding domain based on the local use case sequence belonging to the non-dominated solution can also be achieved in the following ways: when there is only one local use case sequence belonging to the non-dominated solution, the determined local use case sequence belonging to the non-dominated solution is taken as the second reference sequence within the corresponding domain; when the number of local use case sequences belonging to the non-dominated solution is at least two, one local use case sequence is randomly selected from the at least two local use case sequences belonging to the non-dominated solution as the second reference sequence within the corresponding domain.
[0101] In some embodiments, see Figure 6 , Figure 6 This is a flowchart illustrating the steps following step 206 in the embodiments of this application. The test case sorting method provided in the embodiments of this application can also be executed as follows:
[0102] Step 601: Determine the dominance relationship between the first target sequence and the second target sequence based on the fitness function;
[0103] Step 602: Obtain the first target sequence and the use case sequence belonging to the dominant solution in the second target sequence according to the dominance relationship;
[0104] Step 603: Remove the use case sequence belonging to the dominant solution from the preferred sequence set.
[0105] In actual implementation, the terminal can further compare the dominance relationship between the first target sequence and the second target sequence, retain the non-dominated solutions and remove the dominated solutions, thereby further filtering the optimal solution. It should be understood that the optimal solution is the non-dominated solution.
[0106] In some embodiments, the following can also be performed:
[0107] Based on the fitness function, the dominance relationship between the first target sequence and the second target sequence is determined, and the first use case sequence set is updated based on the dominance relationship;
[0108] The target use case sequence set is used as the new second use case sequence set, and the preferred sequence set is updated based on the updated first use case sequence set and the new second use case sequence set.
[0109] When the number of updates to the preferred sequence set reaches the threshold, the preferred sequence set obtained from the last update is obtained, and the test case sequence in the preferred sequence set obtained from the last update is used as the sorting result after sorting the test case set.
[0110] By obtaining a first set of test case sequences corresponding to the test case set used to test the program under test, a fitness function is obtained, and a first target sequence belonging to a non-dominated solution in the first set of test case sequences is determined based on the fitness function; a second set of test case sequences corresponding to the test case set is obtained, and the neighborhood corresponding to each test case sequence in the second set of test case sequences is determined; based on the first target sequence and the neighborhood corresponding to each test case sequence in the second set of test case sequences, a random walk is performed on the second set of test case sequences to obtain a processed target set of test case sequences; based on the fitness function, a second target sequence belonging to a non-dominated solution in the target set of test case sequences is determined; the first target sequence and the second target sequence are added to a preferred sequence set, so that the test case sequences in the preferred sequence set are used as the sorting result after sorting the test case set, which can improve the probability of obtaining the global optimal solution and make the sorting of the test cases more accurate and objective.
[0111] The following will describe an exemplary application of the embodiments of this application in a real-world application scenario. See also... Figure 7 , Figure 7 This is an optional flowchart illustrating the test case sorting method provided in this application embodiment. Specifically,
[0112] Step 1: Define the program coverage record matrix and the test case execution time matrix. The current test case set T = {T1, T2, ..., T...} i ,…,T n} contains n test cases, where T i Let ET be the i-th test case in T. Now, we use the test case set T to test a program containing M statements, denoted as ET = {ET1, ET2, ..., ET...}. i ,...,ET n Let} represent the execution time of all test cases in T, where ET i This represents the time consumed in executing the i-th test case. Test cases are numbered from 1 to n, and statements are numbered from 1 to M. Matrix F has n rows and M columns. If the i-th test case covers the j-th statement, then F... ij =1, otherwise F ij =0.
[0113] Step 2: Selecting sorting criteria and constructing a fitness function. TCP-based regression testing can detect errors earlier than random testing, and the average defect detection rate can be used as a criterion for quantifying the quality of sequences. However, defect detection information is unknown before test case execution, so the average defect detection rate has obvious shortcomings. In general, the greater the coverage of test cases to program statements or branches, the greater the probability of program defects in that test case. Moreover, coverage information can be obtained through coverage analysis tools before test case execution. This invention uses average code coverage and effective test case execution time to construct a fitness function. The fitness function is shown in formulas (1) and (2), and will not be elaborated here.
[0114] Step 3: Define coding rules and initialize the antlion and ant colony, setting the maximum number of iterations; for the test case set T = {T1, T2, ..., T...} i ,...,T n Given a sorting criterion (optimization objective), the essence of the TCP problem is to find a sequence of test cases that optimizes the objective. The antlion algorithm uses two populations: antlion populations and ant populations. Each individual in the population is represented by a 1×n matrix (n is the number of test cases), symbolizing a sorting of the test cases in the test case set. The population size is m, representing the m possible test case execution sequences. Finally, we obtain the m×n dimensional antlion population position matrix AL and ant population position matrix A.
[0115]
[0116] Since individual positions are often initialized as decimals, they cannot represent the order of test cases. This invention uses positional order as the execution order of test cases, taking the i-th antlion position AL as an example. ik For example, in the table [0.63, 0.46, 0.19, 0.25, 0.54, 0.21, 0.95], k represents the test case number, and k = 2 represents the second test case in the test case set, corresponding to position AL. i2 =0.46, corresponding to the order 4, meaning that the execution order of test case 2 in this test case sequence is 4.
[0117]
[0118] Step 4: Obtain the non-dominated set solution set of the initial antlion population and store it in an external archive; calculate the APSC and EET of individual antlions in the initial antlion population, and the antlion AL. i APSC with two fitness values i and EET i Pairwise comparison of dominance relationships between individuals in the population is performed, and non-dominated solutions are stored in an external archive. For the fitness values of the two rankings, the higher the APSC value of the test case ranking sequence, the better, and the lower the EET value, the better. Therefore, when the APSC...i <APSC j And EET i <EET j or APSC i APSC j And EET i >EET j AL i and AL j The solutions are mutually non-dominant and are added to the external archive. When APSC i APSC q And EET i <EET q At that time, in the two optimization objectives, AL i Both are superior to AL q AL i Dominant AL q AL q Unable to be added to the external archive as the optimal solution; set the current iteration count to 1 and start the loop;
[0119] Step 5: Randomly select a non-dominant solution from the external archive as the elite antlion (select a sparser solution based on the distribution of non-dominant solutions). Also, select an antlion from the external archive using the roulette method and dig a trap.
[0120] Step Six: Constructing the individual ant neighborhood and selecting the optimal solution within the neighborhood; In the original ALO algorithm, ants only move based on the antlion and elite antlion selected by roulette wheel, without considering information about their own ant population, resulting in slow convergence. To improve the convergence speed and ensure population diversity, this invention uses a von Neumann topology to construct the individual ant neighborhood, guiding ant movement with the optimal solution within the neighborhood. The specific process of constructing the individual neighborhood based on the von Neumann topology is as follows:
[0121] (1) Arrange the ant population in m rows and n columns, ensuring that pop = m × n;
[0122] (2) For an individual i, i∈{1,2,3,…,pop}, construct the local topology according to the following rules: the upper neighbor is N i (1) = (in)mod pop, if N i (1) = 0, then N i (1) = pop; left neighbor is N i (2) = i-1, if (in)mod pop = 0, then N i (2) = i-1+n; the right neighbor is N i (3) = i + 1, if i mod n = 0, then N i (3) = i + 1 + n; the next neighbor is Ni (4) = (i+n) mod pop, if N i (4) = 0, then N i (4) = pop; the center point is N i (5) = i;
[0123] After the above operations, we can obtain the neighborhood L(i) = {N(1), N(2), N(3), N(4), N(5)} of ant i. Calculate the fitness values of the 5 individuals in the neighborhood, and obtain the non-dominated solutions in step four. If the number of non-dominated solutions is greater than 1, randomly select one as the best individual AntLbest in the neighborhood of ant i. i .
[0124] Step 7: Update the boundary of the ant's random walk; in the ALO algorithm, the ant only updates the boundary of the random walk of ant i in the current iteration according to formula (3);
[0125] Step 8: Update ant positions; First, calculate the random walk distances of the ant around the elite antlion, the antlion selected by roulette wheel, and the best neighboring ant in the current iteration according to formulas (4) and (5), respectively, and then update the ant positions according to formula (6);
[0126] Step 9: Obtain the non-dominated solution set of the current ant population; calculate the fitness values (APSC and EET) of the ant population individuals after the current update position, compare the dominance relationships between the population individuals pairwise, and store the non-dominated solutions to an external archive;
[0127] Step 10: Update the external archive; compare the dominance relationships between individuals in the external archive pairwise, remove the dominant solutions, and keep only the non-dominant solutions; determine if the external archive is saturated, and if so, delete the most densely packed solutions;
[0128] Step 11: Set the mutation probability and use a reverse learning strategy to mutate the ant population. If the population deviates from the optimal solution in the early stages of algorithm iteration, the algorithm will find it difficult to converge to the optimal solution in a short time. This invention uses a reverse learning strategy to generate reverse solutions for partial solutions, optimizes the composition of the ant population, expands the richness of the population solutions, and improves the global search capability of the algorithm. The specific implementation process is as follows: The position of the i-th ant in the n-dimensional search space can be represented as A. i =[A i1 A i2 ,…,A ij ,…,A in Generate a random number c in the range of 0-1. If the random number is less than the set mutation probability, then mutate the individual's position. Where A... ij Let i be the position of the i-th ant in dimension j. and Let these be the maximum and minimum values of ant i across all dimensions in the current iteration:
[0129]
[0130] Step 12: Determine if the iteration termination condition has been met. If the maximum number of iterations has been reached, stop running and output the non-dominated solution set (the set of optimal test case execution sequences) in the external archive. Otherwise, continue to execute steps 5 to 11.
[0131] This application leverages the antlion optimization algorithm as a metaheuristic algorithm for solving TCP problems, demonstrating its excellent performance. By integrating it into the test case sorting process, it effectively calculates the optimal solution. Furthermore, based on the Pareto optimality method, this application obtains a more objective set of test case execution sequences. Additionally, this application employs a reverse learning strategy to mutate some individual ants, ensuring population richness and enabling thorough exploration of the solution region, thus effectively improving the population's global search capability. Moreover, by incorporating an improved antlion optimization algorithm with a von Neumann neighborhood optimality strategy, the application utilizes neighborhood-optimal individuals, elite antlions, and roulette wheel selection to guide ant position updates, accelerating the algorithm's convergence speed and accuracy.
[0132] The following description continues to illustrate the exemplary structure of the test case sorting device 155 provided in the embodiments of this application as a software module. In some embodiments, such as Figure 1 As shown, the software modules in the test case sorting device 155 stored in the memory 150 may include:
[0133] The first acquisition module 1551 is used to acquire the first test case sequence set corresponding to the test case set used to test the program under test;
[0134] The second acquisition module 1552 is used to acquire a fitness function and determine, based on the fitness function, a first target sequence belonging to a non-dominated solution in the first use case sequence set;
[0135] The third obtaining module 1553 is used to obtain the second test case sequence set corresponding to the test case set, and determine the neighborhood corresponding to each test case sequence in the second test case sequence set;
[0136] The random walk processing module 1554 is used to perform random walk processing on the second use case sequence set based on the first target sequence and the neighborhood corresponding to each use case sequence in the second use case sequence set, so as to obtain the processed target use case sequence set.
[0137] The determination module 1555 is used to determine, based on the fitness function, a second target sequence belonging to the non-dominated solution in the target use case sequence set;
[0138] The addition module 1556 is used to add the first target sequence and the second target sequence to the preferred sequence set, so as to use the test case sequence in the preferred sequence set as the sorting result after sorting the test case set.
[0139] In some embodiments, the second obtaining module 1552 is further configured to use the fitness function to determine the fitness value of each use case sequence in the first use case sequence set; wherein the fitness value includes average statement coverage and effective execution time; and based on the fitness value of each use case sequence, to select a first target sequence belonging to a non-dominated solution from the first use case sequence set.
[0140] In some embodiments, the random walk processing module 1554 is further configured to determine the neighborhood corresponding to each use case sequence in the second use case sequence set based on the number of use case sequences in the second use case sequence set and the number of use cases in the test cases set.
[0141] In some embodiments, the number of the first target sequences is multiple; the random walk processing module 1554 is further configured to use a roulette wheel algorithm to select one first target sequence as a first reference sequence from the multiple first target sequences; based on the neighborhood corresponding to each use case sequence in the second use case sequence set, determine a second reference sequence in each neighborhood from the second use case sequence set; and perform random walk processing on the second use case sequence set based on the first reference sequence and the second reference sequence in each neighborhood.
[0142] In some embodiments, the random walk processing module 1554 is further configured to perform the following processing for each use case sequence in the second use case sequence set: based on the neighborhood corresponding to the first use case sequence, determine multiple local use case sequences belonging to the corresponding domain from the second use case sequence set; the first use case sequence is any one of the use case sequences in the second use case sequence set; based on the fitness function, determine the local use case sequences belonging to the non-dominated solution from the multiple local use case sequences; and determine a second reference sequence in the corresponding domain of the local use case sequences belonging to the non-dominated solution.
[0143] In some embodiments, the random walk processing module 1554 is further configured to: when there is only one local use case sequence belonging to a non-dominated solution, use the determined local use case sequence belonging to a non-dominated solution as a second reference sequence in the corresponding domain; when the number of local use case sequences belonging to a non-dominated solution is at least two, randomly select one local use case sequence from the at least two local use case sequences belonging to a non-dominated solution as a second reference sequence in the corresponding domain.
[0144] In some embodiments, the test case sorting device 155 further includes: a sorting module, configured to determine the dominance relationship between the first target sequence and the second target sequence based on the fitness function, and update the first test case sequence set based on the dominance relationship; use the target test case sequence set as a new second test case sequence set, and update the preferred sequence set based on the updated first test case sequence set and the new second test case sequence set; when the number of updates to the preferred sequence set reaches a threshold, obtain the preferred sequence set obtained from the last update, and use the test case sequences in the preferred sequence set obtained from the last update as the sorting result after sorting the test case set.
[0145] This application provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the test case sorting method described above in this application.
[0146] This application provides a computer-readable storage medium storing executable instructions, wherein the executable instructions, when executed by a processor, will cause the processor to execute the test case sorting method provided in this application.
[0147] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.
[0148] In some embodiments, executable instructions may take the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.
[0149] As an example, executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple collaborating files (e.g., a file that stores one or more modules, subroutines, or code sections).
[0150] As an example, executable instructions can be deployed to execute on a single computing device, or on multiple computing devices located in one location, or on multiple computing devices distributed across multiple locations and interconnected via a communication network.
[0151] In summary, the embodiments of this application can increase the probability of obtaining the global optimal solution, making the ranking of the test cases more accurate and objective.
[0152] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.
Claims
1. A method for sorting test cases, characterized in that, The method includes: Obtain the first test case sequence set corresponding to the test case set used to test the program under test; Obtain the fitness function, and based on the fitness function, determine the first target sequence in the first use case sequence set that belongs to the non-dominated solution; Obtain the second test case sequence set corresponding to the test case set, and construct the neighborhood corresponding to each test case sequence in the second test case sequence set according to the von Neumann topology; The test case sequences in the first test case sequence set are treated as antlions, and the test case sequences in the second test case sequence set are treated as ants. Multiple iterations are performed, and for each iteration, the following processing is performed: One first target sequence is randomly selected from multiple first target sequences as the elite antlion; Using the roulette wheel algorithm, one first target sequence is selected as the target antlion from multiple first target sequences; Based on the neighborhood corresponding to each use case sequence in the second use case sequence set, the optimal ant in each neighborhood is determined from the second use case sequence set; For each of the ants, calculate the first random walk position of the ant around the target antlion, the second random walk position of the ant around the elite antlion, and the third random walk position of the ant around the optimal ant. Calculate the average of the first random walk position, the second random walk position, and the third random walk position, and use the average value as the updated position of the ant; In each iteration, the position of each ant in the second use case sequence set is further mutated using the following formula: ； in, For the first Only ants The position of the dimension For the mutated first Only ants The position of the dimension and The first The maximum and minimum values of an ant across all dimensions in the current iteration. c It is a random number. p The set mutation probability; After the multiple iterations are completed, the target use case sequence set is obtained based on the position of each ant in the last update. Based on the fitness function, the second target sequence belonging to the non-dominated solution is determined in the target use case sequence set; The first target sequence and the second target sequence are added to the preferred sequence set, so that the test case sequences in the preferred sequence set are used as the sorting result of the test case set.
2. The method according to claim 1, characterized in that, The step of determining the first target sequence belonging to the non-dominated solution in the first use case sequence set based on the fitness function includes: Using the fitness function, the fitness value of each test case sequence in the first test case sequence set is determined; wherein, the fitness value includes average statement coverage and effective execution time; Based on the fitness value of each use case sequence, a first target sequence belonging to a non-dominated solution is selected from the first use case sequence set.
3. The method according to claim 1, characterized in that, Determining the neighborhood corresponding to each use case sequence in the second use case sequence set includes: Based on the number of test case sequences in the second test case sequence set and the number of test cases in the test case set, the neighborhood corresponding to each test case sequence in the second test case sequence set is determined.
4. The method according to claim 1, characterized in that, The step of determining the optimal ant within each neighborhood of each use case sequence in the second use case sequence set includes: For each use case sequence in the second use case sequence set, perform the following processing: Based on the neighborhood corresponding to the first use case sequence, multiple local use case sequences belonging to the corresponding neighborhood are determined from the second use case sequence set; the first use case sequence is any one of the use case sequences in the second use case sequence set. Based on the fitness function, determine the local use case sequence that belongs to the non-dominated solution from the plurality of local use case sequences; Determine the optimal ant in the corresponding domain of the local use case sequence that belongs to the non-dominated solution.
5. The method according to claim 4, characterized in that, Determining the optimal ant within the corresponding neighborhood of the local use case sequence belonging to the non-dominated solution includes: When there is only one local use case sequence belonging to the non-dominated solution, the determined local use case sequence belonging to the non-dominated solution is taken as the optimal ant in the corresponding neighborhood. When there are at least two local use case sequences belonging to non-dominated solutions, one local use case sequence is randomly selected from the at least two local use case sequences belonging to non-dominated solutions as the optimal ant in the corresponding neighborhood.
6. The method according to claim 1, characterized in that, The method further includes: The dominance relationship between the first target sequence and the second target sequence is determined based on the fitness function; the use case sequences belonging to the dominant solutions in the first target sequence and the second target sequence are obtained according to the dominance relationship; The use case sequences belonging to the dominant solution are removed from the preferred sequence set.
7. The method according to claim 1, characterized in that, The method further includes: Based on the fitness function, the dominance relationship between the first target sequence and the second target sequence is determined, and the first use case sequence set is updated based on the dominance relationship; The target use case sequence set is used as the new second use case sequence set, and the preferred sequence set is updated based on the updated first use case sequence set and the new second use case sequence set. When the number of updates to the preferred sequence set reaches the threshold, the preferred sequence set obtained from the last update is obtained, and the test case sequence in the preferred sequence set obtained from the last update is used as the sorting result after sorting the test case set.
8. A test case sorting device, characterized in that, include: The first acquisition module is used to acquire the first test case sequence set corresponding to the test case set used to test the program under test; The second acquisition module is used to acquire a fitness function and determine, based on the fitness function, a first target sequence belonging to a non-dominated solution in the first use case sequence set; The third acquisition module is used to acquire the second test case sequence set corresponding to the test case set, and construct the neighborhood corresponding to each test case sequence in the second test case sequence set according to the von Neumann topology. The random walk processing module is used to treat the test case sequences in the first test case sequence set as antlions and the test case sequences in the second test case sequence set as ants, perform multiple iterations, and perform the following processing for each iteration: One first target sequence is randomly selected from multiple first target sequences as the elite antlion; Using the roulette wheel algorithm, one first target sequence is selected as the target antlion from multiple first target sequences; Based on the neighborhood corresponding to each use case sequence in the second use case sequence set, the optimal ant in each neighborhood is determined from the second use case sequence set; For each of the ants, calculate the first random walk position of the ant around the target antlion, the second random walk position of the ant around the elite antlion, and the third random walk position of the ant around the optimal ant. Calculate the average of the first random walk position, the second random walk position, and the third random walk position, and use the average value as the updated position of the ant; In each iteration, the position of each ant in the second use case sequence set is further mutated using the following formula: ； in, For the first Only ants The position of the dimension For the mutated first Only ants The position of the dimension and The first The maximum and minimum values of an ant across all dimensions in the current iteration. c It is a random number. p The set mutation probability; After the multiple iterations are completed, the target use case sequence set is obtained based on the position of each ant in the last update. The determination module is used to determine, based on the fitness function, a second target sequence belonging to the non-dominated solution in the target use case sequence set; An adding module is used to add the first target sequence and the second target sequence to the preferred sequence set, so that the test case sequences in the preferred sequence set are used as the sorting result after sorting the test case set.
9. An electronic device, characterized in that, include: Memory, used to store executable instructions; A processor, when executing executable instructions stored in the memory, implements the test case sorting method according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, It stores executable instructions for implementing the test case sorting method according to any one of claims 1 to 7 when executed by a processor.