Test case processing method, device, equipment, program product and storage medium

Abstract

The application provides a test case processing method, device, equipment, program product and storage medium. The method comprises the following steps: obtaining a candidate test case set and test requirement information, wherein the candidate test case set comprises a plurality of candidate test cases; extracting target test cases meeting the test requirement information from the candidate test case set; determining an execution order of each target test case according to historical test data; and executing a test process according to the execution order and each target test case. Through the application, the execution efficiency of test cases in the test process can be improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method, apparatus, device, program product, and storage medium for processing test cases. Background Technology

[0002] When performing software and system integration testing on vehicle controllers, three common methods are used: full-function testing, requirement change testing, and smoke testing. Among these, full-function testing has a long testing cycle, is time-consuming and labor-intensive, requiring significant resources and personnel, and in a rapidly iterating development environment, it's difficult to guarantee release time. Requirement change testing, especially with large-scale software, cannot assess the impact of code changes on overall functionality, resulting in insufficient test coverage. Smoke testing has limited coverage, carries the risk of missing defects, is unsuitable for in-depth problem diagnosis, and is highly dependent on the selection of test cases.

[0003] Currently, there is no more efficient way to improve the execution efficiency of test cases during the testing process in the relevant technologies. Summary of the Invention

[0004] This application provides a method, apparatus, device, program product, and storage medium for processing test cases, which can improve the execution efficiency of test cases during the testing process.

[0005] The technical solution of this application embodiment is implemented as follows:

[0006] This application provides a method for processing test cases, the method comprising:

[0007] Obtain a set of candidate test cases and test requirement information, wherein the set of candidate test cases includes multiple candidate test cases;

[0008] Extract target test cases that satisfy the test requirements from the set of candidate test cases;

[0009] The execution order of each target test case is determined based on historical test data;

[0010] The testing process is executed according to the execution order and each of the target test cases.

[0011] This application provides a test case processing apparatus, comprising:

[0012] The test case extraction module is used to obtain a set of candidate test cases and test requirement information, wherein the set of candidate test cases includes multiple candidate test cases; and to extract target test cases that meet the test requirement information from the set of candidate test cases.

[0013] The test case sorting module is used to determine the execution order of each target test case based on historical test data; and to execute the test process according to the execution order and each target test case.

[0014] This application provides an electronic device, the electronic device comprising:

[0015] Memory is used to store executable instructions or computer programs.

[0016] The processor, when executing computer-executable instructions stored in the memory, implements the test case processing method provided in the embodiments of this application.

[0017] This application provides a computer-readable storage medium storing a computer program or computer-executable instructions for implementing the test case processing method provided in this application when executed by a processor.

[0018] This application provides a computer program product, including a computer program or computer executable instructions. When the computer program or computer executable instructions are executed by a processor, they implement the test case processing method provided in this application.

[0019] The embodiments of this application have the following beneficial effects:

[0020] By acquiring a set of candidate test cases and test requirement information, target test cases that meet the test requirements are extracted from the candidate set, ensuring that only test cases that meet the requirements are selected, saving time and resources. The execution order of each target test case is determined based on historical test data. Then, the test process is executed according to the execution order and each target test case, rationally allocating test resources, flexibly adjusting tests according to requirements, and specifically executing tests according to the execution order, improving the effectiveness and efficiency of testing, thereby saving the computational resources and time required for testing. Attached Figure Description

[0021] Figure 1 This is a schematic diagram illustrating the application mode of the test case processing method provided in the embodiments of this application;

[0022] Figure 2 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application;

[0023] Figure 3A This is a first flowchart illustrating the test case processing method provided in the embodiments of this application;

[0024] Figure 3B This is a second flowchart illustrating the test case processing method provided in the embodiments of this application;

[0025] Figure 3C This is a schematic diagram of the third process of the test case processing method provided in the embodiments of this application;

[0026] Figure 4 This is a schematic diagram of the fourth process of the test case processing method provided in the embodiments of this application.

[0027] It should be noted that the terms "first" and "second" mentioned above are only used to distinguish between different options and do not represent the degree of superiority or inferiority of the options or their priority in the implementation process. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0029] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0030] In the following description, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0031] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0032] Unless otherwise defined, all technical and scientific terms used in the embodiments of this application have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in the embodiments of this application is for the purpose of describing the embodiments of this application only and is not intended to limit this application.

[0033] In this application embodiment, the collection and processing of relevant data (e.g., test requirements and test case data) should strictly comply with the requirements of relevant national laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the personal information subject.

[0034] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0035] 1) Smoke Testing: This is a rapid basic functionality verification strategy in the software development process. It's a means of confirming and verifying the basic functions of the software, not an in-depth test of the software version package. Smoke testing is also a pre-test before detailed testing of the software version package. The main purpose of performing smoke testing is to quickly verify whether there are defects in the basic functions of the software. If the smoke test cases fail, further testing is unnecessary. Before conducting smoke testing, it is necessary to determine the set of smoke test cases, which should cover the basic functions of the software.

[0036] 2) Test requirements: These refer to the specific conditions and standards that must be met when testing a software product or system. They define the goals and scope of testing activities, ensuring that testing comprehensively covers all important functional and non-functional requirements. Functional requirements describe what the system should do and are related to specific functions of the system, while non-functional requirements describe how the system should operate.

[0037] 3) Test Case Prioritization (TCP): This is a test management technique that determines the execution order of test cases to make more efficient use of test resources and time. Under limited time and resource constraints, prioritizing the execution of certain test cases can help discover critical defects and make testing more purposeful.

[0038] 4) Test case execution status (ES): refers to the different stages or results of the test case during execution, providing information about whether the test case has passed, failed, or is in other states. In this embodiment, the test case execution status includes: execution passed, execution failed, and not executed.

[0039] 5) Test Case Execution Duration (DT): This refers to the time elapsed from the start to the end of a test case execution during the testing process. This metric is crucial for evaluating test efficiency, optimizing the test process, and understanding the complexity of test cases. Comparing the execution time of a test case with similar test cases can help identify potential efficiency issues.

[0040] This application provides a method for processing test cases, an apparatus for processing test cases, an electronic device, a computer-readable storage medium, and a computer program product, which can improve the execution efficiency of test cases during the testing process.

[0041] The following describes exemplary applications of the electronic devices provided in the embodiments of this application. These electronic devices can be implemented as various types of terminals such as laptops, tablets, desktop computers, set-top boxes, smartphones, smart speakers, smartwatches, smart TVs, and in-vehicle terminals, or as servers. The following will describe exemplary applications when the device is implemented as an in-vehicle terminal.

[0042] See Figure 1 , Figure 1 This is a schematic diagram illustrating the application mode of the test case processing method provided in the embodiments of this application. It is an example to support the processing of one test case. Figure 1 The system involves server 200, network 300, terminal device 400 and database 500. Terminal device 400 is connected to server 200 through network 300. Network 300 can be a wide area network or a local area network, or a combination of both.

[0043] In some embodiments, the user is a tester, the server 200 is a server that sends historical test cases to the terminal device 400 in response to a test request, the terminal device 400 may be the tester's computer, the terminal device 400 is used to execute the test case processing method in response to the test request, and the database 500 stores historical test cases.

[0044] For example, a user triggers terminal device 400 to send a test request. Terminal device 400 sends the test request to server 200 via network 300. Server 200 retrieves historical test cases from database 500 and sends the historical test cases to terminal device 400 via network 300. Terminal device 400 filters candidate test cases based on test requirements, sorts the filtered test cases according to the historical test case data sent by the server, determines the execution order of candidate test cases, and executes the test task purposefully by executing the set of test cases with the execution order.

[0045] In some embodiments, the test case processing method of this application can also be applied to the following application scenarios:

[0046] This involves processing test cases in different scenarios and determining their execution order. For example, a test scenario may include multiple test cases, and as test requirements iterate and increase, the number of test cases for the application under test also gradually increases. In related technologies, full-featured testing has a long testing cycle, is time-consuming and labor-intensive, requires significant resources and personnel, and makes it difficult to guarantee release time. Change testing suffers from insufficient coverage when the software is large-scale, and smoke testing has limited coverage and carries the risk of missing defects.

[0047] The test case processing method of this application embodiment can extract target test cases that meet test requirements from the candidate test case set, ensuring that test requirements are met, filtering out test cases that meet the requirements, saving time and resources, determining the execution order of each target test case based on historical test data, reasonably allocating test resources, and executing tests in a targeted manner according to the execution order, thereby improving the execution efficiency of test cases during the testing process.

[0048] See Figure 2 , Figure 2 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Figure 2 The electronic device shown may be Figure 1 The terminal device 400 includes at least one processor 410, a memory 450, at least one network interface 420, and a user interface 430. The various components in server 200 are coupled together via a bus system 440. It is understood that the bus system 440 is used to implement communication between these components. In addition to a data bus, the bus system 440 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in… Figure 2 The general labeled all buses as Bus System 440.

[0049] The processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0050] User interface 430 includes one or more output devices 431 that enable the presentation of media content, including one or more speakers and / or one or more visual displays. User interface 430 also includes one or more input devices 432, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0051] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The memory 450 may optionally include one or more storage devices physically located away from the processor 410.

[0052] The memory 450 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 450 described in this application embodiment is intended to include any suitable type of memory.

[0053] In some embodiments, memory 450 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0054] Operating system 451 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks;

[0055] The network communication module 452 is used to reach other electronic devices via one or more (wired or wireless) network interfaces 420, exemplary network interfaces 420 including: Bluetooth, WiFi, and Universal Serial Bus (USB), etc.

[0056] Presentation module 453 is configured to enable the presentation of information (e.g., a user interface for operating peripheral devices and displaying content and information) via one or more output devices 431 (e.g., a display screen, a speaker, etc.) associated with user interface 430;

[0057] The input processing module 454 is used to detect and translate one or more user inputs or interactions from one or more input devices 432.

[0058] In some embodiments, the apparatus provided in this application can be implemented in software. Figure 2 A processing device 455 for test cases stored in memory 450 is shown. This device can be software in the form of programs or plug-ins, and includes the following software modules: a test case extraction module 4551 and a test case sorting module 4552. These modules are logically linked and can therefore be arbitrarily combined or further split according to their implemented functions. The functions of each module will be described below.

[0059] The method for processing test cases provided in the embodiments of this application will be described in conjunction with exemplary applications and implementations of the electronic devices provided in the embodiments of this application.

[0060] The following describes the test case processing method provided in the embodiments of this application. As mentioned above, the electronic device implementing the device management method of the embodiments of this application can be a terminal, a server, or a combination of both. Therefore, the executing entity of each step will not be described again below.

[0061] See Figure 3A , Figure 3A This is a first flowchart illustrating the test case processing method provided in this application embodiment, which will be combined with... Figure 3A The steps shown are explained below. Figure 3A The implementing body is Figure 1 Terminal device 400.

[0062] In step 301, a set of candidate test cases and test requirement information are obtained.

[0063] Here, the candidate test case set includes multiple candidate test cases.

[0064] For example, test requirement information is determined before testing based on a pre-configured test requirement document by technical personnel. This information defines the specific conditions and functional behaviors that test cases must meet. Test case data is extracted from the candidate test case set. Each candidate test case includes: its code, requirement code, test level, test case change information, and requirement change requirements. Table 1 describes the test case data, as detailed below.

[0065] Table 1

[0066] Test case number Requirement Number Test Level Use Case Changes Requirement Change <![CDATA[TC1]]> <![CDATA[Req 01 ,Req 02 ]]> A 0 1 … … … … … <![CDATA[TC n ]]> <![CDATA[Req i ]]> D 1 1

[0067] Among them, the test case number (TC) i ) represents the candidate test case number in the candidate test case set, and the requirement number (Req) i The `test_requirement_number` represents the software requirement number traced by the candidate test case, and the `test_level` represents the severity level of the candidate test case's failure. `test_case_change` indicates whether the candidate test case has been updated compared to the previous version; a value of 0 indicates no change, and a value of 1 indicates an update. `requirement_change` indicates whether the test requirements covered by the candidate test case have been updated compared to the previous version; a value of 0 indicates no change, and a value of 1 indicates an update.

[0068] In step 302, target test cases that meet the test requirements information are extracted from the candidate test case set.

[0069] Here, the test requirement information includes: a set of change requirements and a set of total requirements. The set of total requirements includes multiple test requirements, and the set of change requirements is a subset of the set of total requirements.

[0070] For example, the requirements corresponding to candidate test cases that have been updated compared to the previous version are stored in the change requirement set. Candidate test cases that meet the change requirements are extracted from the candidate test case set and used as target test cases.

[0071] In some embodiments, see Figure 3B , Figure 3B This is a second flowchart illustrating the test case processing method provided in the embodiments of this application; Figure 3A Step 302 can be executed Figure 3B Steps 3021 to 3024 are implemented, and the details are explained below.

[0072] In step 3021, a first set of test cases that meets the test requirements in the change requirement set is extracted from the candidate test case set.

[0073] Here, the first test case set includes candidate test cases that satisfy the test requirements in the change requirement set, and the first test case set is the smallest set that satisfies the change requirement set.

[0074] For example, initialize a candidate test case set to store the first test case that meets the test requirements. According to the test requirements, add all test cases that directly cover the change requirements to the candidate test case set. This can be achieved by formula (1), which will be explained in detail below.

[0075]

[0076] Here, TC represents the candidate test case set, which includes multiple candidate test cases, TC = {TC1, TC2, ..., TCn}; Req represents the total requirement set, which includes multiple test requirements, Req = {Req1, Req2, ..., Reqm}; and ChgReq represents the change requirement set, which is the set of requirements corresponding to the candidate test cases that have changed compared to the previous version, ChgReq = {Reqk, Req{k+1}, ..., Reqj}∈Req.

[0077] If C(TC) i The intersection of ) and ChgReq is not empty (i.e. ), indicating the candidate test case TC iCandidate test cases (TCs) that cover at least one change requirement will be provided, covering at least one requirement in the change requirement set (ChgReq). i Add the test cases to the first test case set (TCS). The first test case set is the minimum set that satisfies the change requirements set. That is, a minimum smoke test case set is generated by extracting test cases that satisfy the test requirements in the change requirements set from the candidate test case set. Smoke testing is a pre-test before detailed testing. The main purpose of performing smoke testing is to quickly verify whether there are defects in the basic functions of the software. If the test cases in the smoke test fail, no further testing is necessary. The minimum smoke test case set is the set with the fewest test cases that satisfy the smoke test.

[0078] In step 3022, the total demand set is subtracted from the change demand set to obtain the first demand set.

[0079] For example, subtracting the change requirement set from the total requirement set yields the test requirements not covered by the test cases in the current first test case set, which is then used as the first requirement set. The first requirement set can also be called the remaining requirement set; that is, the first requirement set is the set of unmet remaining test requirements of the candidate test cases in the first test case set.

[0080] In step 3023, from the remaining candidate test cases after subtracting the first test case set from the candidate test case set, all candidate test cases that satisfy the first requirement set are selected and added to the first test case set to form the second test case set.

[0081] For example, if the first requirement set, obtained by subtracting the total requirement set from the change requirement set, is not empty, it indicates that the test cases in the first test case set do not completely cover all test requirements. From the remaining test cases in the candidate test case set, the test cases that cover the most remaining requirements are selected and iteratively added to the first test case set to form the second test case set. The candidate test cases in the second test case set are used as target test cases.

[0082] In some embodiments, Figure 3BStep 3023 can be implemented as follows: Based on the first requirement set, determine the remaining requirement coverage of the candidate test cases remaining after subtracting the first test case set from the candidate test case set, where the remaining requirement coverage is the number of test requirements in the first requirement set satisfied by the candidate test cases; select the candidate test case corresponding to the largest remaining requirement coverage and add it to the first test case set to form an updated first test case set, and determine the second requirement set composed of the test requirements satisfied by the candidate test cases in the updated first test case set; when the second requirement set is different from the total requirement set, update the first requirement set according to the unselected candidate test cases, and perform the step of "determining the remaining requirement coverage of the remaining candidate test cases in the candidate test case set based on the first requirement set" based on the updated first test case set; when the second requirement set is the same as the total requirement set, use the updated first test case set as the second test case set.

[0083] For example, if the test cases in the first test case set do not completely cover all test requirements, calculate the test case threshold (TC) for each unselected candidate test case based on the first requirement set. i The remaining demand that can be covered | C(TC) i )-CoveredReq|。 The candidate test cases corresponding to the largest number of remaining requirements covered can be selected by formula (3), which is explained in detail below.

[0084]

[0085] Wherein, the max function represents finding the maximum value, TC_{max} represents the candidate test case that can cover the most remaining requirements, and in formula (3), CoveredReq represents the set of change requirements (the initial second set of requirements), which is used to select a test case that can cover the most remaining requirements from all unselected candidate test cases. For all test cases not in the first test set, TC i Select a test case TC_{max} that can cover the maximum remaining requirements.

[0086] For example, select a test case TC_{max} that can cover the most remaining requirements, add it to the first test set for updating, and form the updated first test case set. The second requirement set, which consists of the test requirements satisfied by the candidate test cases in the updated first test case set, can be achieved by formulas (4) and (5), which are explained in detail below.

[0087] TCS2=TCS1∪TC max (4)

[0088] CoveredReq2=CoveredReq1∪C(TC {max} (5)

[0089] Among them, the old first test case set (TCS1) is combined with the test case set TC that can cover the most remaining requirements. {max} Merge to obtain the second test case set (TCS2), and combine the initial second requirement set (CoveredReq1) with the test cases TC that can cover the most remaining requirements. {max} The satisfied test requirements are merged to obtain a new second set of requirements (CoveredReq2). The candidate test case set (TCS1) and the second set of requirements (CoveredReq1) are updated iteratively until the second set of requirements is equal to the total set of requirements, that is, CoveredReq2 = Req. The updated first set of test cases is then used as the second set of test cases.

[0090] In step 3024, candidate test cases in the second test case set are selected as target test cases.

[0091] For example, the candidate test cases in the second test case set can satisfy all the requirements in the total requirement set, and the second test case set is the set with the fewest number of candidate test cases that can satisfy all the requirements. All candidate test cases in the second test case set are used as target test cases.

[0092] In this embodiment, by extracting a first set of test cases that meets the change requirements from the candidate test case set, smoke testing can be performed quickly, avoiding unnecessary detailed testing and saving time and resources. By iteratively updating the first set of test cases to form a second set of test cases, it is ensured that the test cases can fully cover all testing requirements, achieving the highest test coverage with the fewest possible test cases.

[0093] See also Figure 3A In step 303, the execution order of each target test case is determined based on historical test data.

[0094] Here, historical test data includes: the execution status and test case parameters for each target test case.

[0095] For example, importing a historical test data set to obtain historical test case data, and determining the execution order of target test cases based on the execution status type and test case parameters of each target test case in the historical test data. Test case execution status refers to the different stages or results a test case is in during execution, providing information about whether the test case has passed, failed, or is in other states. In this embodiment, the types of test case execution status include: execution passed, execution failed, and not executed. Execution passed indicates that the test case was executed successfully, meeting all expected results; execution failed indicates that the test case was not executed successfully, failing to meet one or more expected results; not executed indicates that the test case has not yet started execution. Test case execution duration refers to the time elapsed from the start to the end of test case execution during the testing process. The historical test case data can be described in Table 2, as detailed below.

[0096] Table 2

[0097] Test case number Execution state (i, 1) Execution status Execution state (i, m) Time consumed (DT) <![CDATA[TC1]]> 1 (Execution Successful) … -1 (Execution failed) 104 … … … … … <![CDATA[TC n ]]> 0 (Not executed) … 1 49

[0098] When the execution status value is 1, it represents the historical test case TC. i When the execution status of the j-th test loop is "passed" (or 0), it represents the historical test case TC. i When the execution status of the j-th test loop is "not executed" and the execution status value is -1, it represents the historical test case TC. i The execution status of the j-th test loop is failure, and the execution time (DT) represents the duration of the test case.

[0099] For example, test case parameters are a specific set of information or data used when executing test cases to define and differentiate test case characteristics. Execution status and test case parameters together determine the execution order of the target test cases.

[0100] In some embodiments, see Figure 3C , Figure 3C This is a schematic diagram of the third process of the test case processing method provided in the embodiments of this application; Figure 3A Step 303 can be executed Figure 3C Steps 3031 to 3034 are implemented, and the details are explained below.

[0101] In step 3031, the target test cases are divided into multiple test case groups according to the type of execution state corresponding to each target test case.

[0102] Here, the execution status of the target test cases in each test case group is the same.

[0103] For example, the execution status types include: execution passed, execution failed, and not executed. Based on the execution status type corresponding to each target test case, the target test cases are divided into groups of test cases that have passed, test cases that have failed, and test cases that have not been executed.

[0104] In some embodiments, Figure 3C Step 3031 is also implemented by the following method: traversing each target test case and adding the traversed target test cases to the test case group corresponding to the execution status type, wherein the execution status types include: not executed, executed successfully, and executed failed.

[0105] For example, iterate through each target test case and add the test case to the test case group corresponding to each state type based on the execution status in the target test case data.

[0106] In step 3032, the execution order between test case groups is determined based on the priority of the execution status type.

[0107] For example, execution status types have priorities, which decrease in the order of unexecuted test cases, failed test cases, and successful test cases. The execution order of each test case group is determined as follows: test case groups with an unexecuted status type are executed first, then test case groups with a failed status type are executed, and finally test case groups with a successful status type are executed.

[0108] In step 3033, the execution order of the target test cases in each test case group is determined based on the test case parameters.

[0109] Here, the test case parameters include: test case execution duration and requirement association values.

[0110] For example, the target test cases in each test case group have the same execution state type. After determining the execution order between groups, the execution order of test cases within a group is determined based on the test case parameters. Test cases with higher requirement correlation values ​​are executed first. When the requirement correlation values ​​are the same, test cases with shorter durations are executed first. The process of determining the order within a group is explained in detail below.

[0111] In some embodiments, Figure 3C Step 3033 can be implemented by the following method: Perform the following processing for each test case group: Sort each target test case in descending order according to the requirement association value of each target test case to obtain the first order; Adjust the order of multiple target test cases with the same requirement association value according to the execution time of the test cases to obtain the execution order within the group, wherein the adjustment is based on the ascending order of the execution time of the test cases.

[0112] For example, each target test case has a requirement association value, which is the strength of the association between the current target test case and historical test cases. Test cases are sorted in descending order based on their requirement association values, and test cases with higher requirement association values ​​are executed first. If test cases have the same requirement association value, they are sorted in ascending order based on their execution time, and test cases with shorter execution times are executed first.

[0113] In step 3034, the execution order between each target test case is determined based on the inter-group execution order and the intra-group execution order.

[0114] For example, the execution order between target test cases can be determined by formula (6) based on the order between groups determined by the priority of execution status type and the execution order of test cases within groups determined by the requirement association value and execution duration. The specific explanation is as follows.

[0115]

[0116] Where T″″ and T′ represent the relative order of test cases in the execution order p. In the set of all possible test case execution orders P, for any test case execution order T″, if another test case execution order T′ exists, then the priority of T′ must be higher than or equal to the priority of T″. Target test cases T′ with execution status types of failure or not executed should be executed before any other test case T″. f is a priority function that assigns a priority value to test case T based on its execution status type (pass / fail / not executed). The execution order T″ of all test cases in set P is represented by (T″∈P), where T″ is an element of set P. T″≠T′ represents comparing the execution orders of two different test cases when T″ is not equal to T′. [f(T′)≥f(T″)] represents determining the optimal priority order of candidate test case sets by comparing the values ​​of f for any two test case execution orders T′ and T″.

[0117] In this embodiment of the application, by determining the execution order of test cases based on historical test data, test cases that are most likely to discover defects can be executed first, test resources can be utilized more effectively, and test execution can be performed more purposefully by considering the correlation value of test case requirements and execution time, thereby improving the efficiency and effectiveness of software testing.

[0118] See also Figure 3A In step 304, the test process is executed according to the execution order and each target test case.

[0119] For example, based on each selected target test case, the test process is executed in the determined execution order of the test cases to achieve comprehensive testing of the system functions. Within the same execution time, the purpose of the test is made clearer, and the effectiveness of the test is improved.

[0120] In this embodiment, by selecting a first set of test cases that meets the changed requirements, the purposefulness of testing is enhanced, testing of unchanged parts is reduced, testing costs are lowered, and resource utilization efficiency is improved. By iteratively updating the test case set until all test requirements are covered, the comprehensiveness of the testing work is ensured, and omissions of requirements are avoided. Based on historical test data and requirement changes, the execution order of test cases is determined, with failed test cases executed first, leading to faster discovery of software defects and improved testing effectiveness. Within the same execution time, the testing work has a clearer objective and is more efficient.

[0121] The following will describe an exemplary application of the embodiments of this application in a real-world application scenario.

[0122] In actual vehicle controller testing, when implementing software and system integration testing schemes for vehicle controllers, three main methods are typically used: full-function testing, requirement change testing, and smoke testing. Full-function testing comprehensively tests the vehicle controller to ensure all functions work as expected, covering all functional points of the system. However, it is time-consuming, labor-intensive, and requires significant resources and personnel, making it difficult to guarantee release time in rapidly iterating development environments. Requirement change testing is conducted after software requirements have changed. In large-scale software applications, it's difficult to assess the impact of code changes on overall functionality, resulting in insufficient test coverage. Smoke testing is performed during the transition from development to testing, is relatively short, and primarily targets requirement coverage and main functions. However, its coverage is limited, carries the risk of missing defects, is unsuitable for in-depth problem diagnosis, and is highly dependent on the selection of test cases.

[0123] This application's embodiments prioritize test cases using historical test data, helping to focus on the highest-risk, most likely-to-problem test cases. This allows for the discovery of critical defects in the early stages of continuous integration, enabling rapid fixes with minimal resource consumption. The prioritization method can dynamically adjust test coverage based on the project's current state, ensuring coverage of the most critical functions and paths within a limited testing cycle, making testing more purposeful.

[0124] The following explanation is in conjunction with the accompanying drawings. Figure 4 , Figure 4 This is a schematic diagram of the fourth process of the test case processing method provided in this application embodiment. The execution subject can be a terminal device, a server, or a combination of both. This application embodiment takes a terminal device as the execution subject as an example, and will combine... Figure 4 The steps shown are explained in detail.

[0125] In step 401, test case data is read.

[0126] For example, this involves reading test case data from a test case dataset, which contains multiple test cases. Each test case includes: test case code, requirement code, test level, test case change information, and requirement changes. Table 1 describes the test case data, as detailed below.

[0127] Table 1

[0128] Test case number Requirement Number Test Level Use Case Changes Requirement Change <![CDATA[TC1]]> <![CDATA[Req 01 ,Req 02 ]]> A 0 1 … … … … … <![CDATA[TC n ]]> <![CDATA[Req i ]]> D 1 1

[0129] Among them, the test case number (TC) i ) represents the test case number in the test case set, and the requirement number (Req) i The test case number represents the software requirement number traced by the test case. The test level represents the severity level of the test case failure. The test case change represents whether the test case has been updated compared to the previous version. A value of 0 indicates that the test case has not changed compared to the previous version, while a value of 1 indicates that the test case has been updated. The requirement change represents whether the test requirements covered by the test case have been updated compared to the previous version. A value of 0 indicates that the test requirements covered by the test case have not changed compared to the previous version, while a value of 1 indicates that the test requirements covered by the test case have been updated.

[0130] In step 402, test cases are filtered based on the remaining requirements, and the candidate test case set is updated.

[0131] For example, a candidate test case set is initialized to store the selected test cases. Based on the target requirements, all test cases that directly cover the changed requirements are added first to generate the candidate test case set. The process of generating the candidate test case set is for the purpose of minimizing the smoke test case set. Smoke testing is a pre-test before detailed testing. The main purpose of performing smoke testing is to quickly verify whether there are defects in the basic functions of the software. If the test cases in the smoke test fail, no further testing is necessary. Before performing smoke testing, it is necessary to determine the set of test cases for smoke testing. The set of test cases must cover the basic functions of the software. The minimum smoke test case set is the set with the fewest test cases that satisfy the smoke test. It can be achieved by formula (1), which is explained in detail below.

[0132]

[0133] Where TC represents the test case set, including multiple test cases, TC = {TC1, TC2, ..., TCn}; Req represents the requirement set, including multiple target requirements, Req = {Req1, Req2, ..., Reqm}; and ChgReq represents the change requirement set, which is the set of requirements corresponding to the test cases that have changed compared to the previous version, ChgReg = {Reqk, Req{k+1}, ..., Reqj} ∈ Req. If C(TC i The intersection of ) and ChgReq is not empty (i.e. ), indicating the test case TC i Test cases (TCs) that cover at least one change requirement will be provided, covering at least one requirement in the change requirement set (ChgReq). i Add it to the candidate test case set (TCS).

[0134] In step 403, it is determined whether the candidate test case set covers all target requirements.

[0135] If the result of the judgment in step 403 is negative, continue to execute step 402.

[0136] For example, after adding test cases that cover the requirements to the candidate test case set, updating the covered requirement set (CoveredReq) can be achieved through formula (2), which is explained in detail below.

[0137] CoveredReq=CoveredReq∪∪_{TC i ∈TCS}C(TC i (2)

[0138] CoveredReq represents the set of requirements that have been covered, including the requirements covered in the candidate test case set, C(TC) i ) indicates test case TC i The covered set of requirements, where ChgReq represents the set of requirements that have changed, and for each test case in the candidate test case set, TC i Test cases TC i The covered demand set C(TC) i Add the candidate test case set to the covered requirement set, merge the requirement sets covered by the candidate test case set, and update the covered requirement set.

[0139] For example, when the covered requirement set does not cover all target requirements (i.e., CoveredReq ≠ Req), test cases are selected and added according to a greedy strategy, and the test case count (TC) is calculated for each unselected test case. i The remaining demand that can be covered | C(TC) i)-CoveredReq|, select the test case TC_{max} that can cover the most remaining requirements and add it to the candidate test case set. This can be achieved by formula (3), which is explained in detail below.

[0140]

[0141] Here, the `max` function represents finding the maximum value, `TC_{max}` represents the test case that covers the most remaining requirements, and `CoveredReq` represents the set of covered requirements. It is used to select a test case that covers the most remaining requirements from all unselected test cases. This is for all test cases not in `TCS`. i Select a test case TC_{max} that can cover the most remaining requirements. The selection process is greedy, that is, it always selects the test case that seems to be the best at the moment.

[0142] For example, after adding the test case TC{max} that can cover the most remaining requirements to the candidate test case set, the candidate test case set and the set of covered requirements can be updated using formulas (4) and (5), which are explained in detail below.

[0143] TCS2=TCS1∪TC max (4)

[0144] CoveredReq2=CoveredReq1∪C(TC {max} (5)

[0145] Among them, the old candidate test case set (TCS1) is combined with the test case set TC that can cover the most remaining requirements. {max} Merge the results to obtain a new candidate test case set (TCS2), and combine the old covered requirement set (CoveredReq1) with the test cases TC that cover the most remaining requirements. {max} The satisfied test requirements are merged to obtain a new set of covered requirements (CoveredReq2). The candidate test case set (TCS1) and the set of covered requirements (CoveredReq1) are updated repeatedly until the set of covered requirements is equal to the target set of requirements, that is, CoveredReq2 = Req. At this point, the candidate test case set is the smallest set of candidate test cases that covers all requirements and contains test cases corresponding to all changed requirements.

[0146] If the judgment result in step 403 is yes, then step 404 is executed, in which the historical test dataset is imported.

[0147] For example, importing a historical test dataset retrieves data on historical test cases, including the execution status and execution duration of each test case. The execution status refers to the different stages or results a test case is in during execution, providing information about whether the test case has passed, failed, or is in another state. In this embodiment, the execution status types include: passed, failed, and not executed. Passed execution indicates that the test case was executed successfully, meeting all expected results; failed execution indicates that the test case was not executed successfully, failing to meet one or more expected results; and not executed indicates that the test case has not yet started execution. The execution duration refers to the time elapsed from the start to the end of the test case execution. The historical test case dataset can be described in Table 2, as detailed below.

[0148] Table 2

[0149]

[0150] When the execution status value is 1, it represents the test case TC. i When the execution status of the j-th test loop is "passed" (or 0), it indicates that the test case is TC (Test Case Troubleshooting). i When the execution status of the j-th test loop is "not executed" and the execution status value is -1, it represents that the test case TC is not executed. i The execution status of the j-th test loop is failure, and the execution time (DT) represents the duration of the test case.

[0151] In step 405, the priority ranking of the candidate test case set is determined based on the historical test dataset.

[0152] For example, based on the historical test dataset, the test case filtering order is defined according to the test execution status (pass / fail / not executed) in the historical test dataset. The priority ranking of the candidate test case set is determined according to the test case filtering order, which can be achieved by formula (6), as explained in detail below.

[0153]

[0154] Where T″ and T′ represent the relative order of test cases in the execution order p. In the set of all possible test case execution orders P, for any test case execution order T″, if another test case execution order T′ exists, then the priority of T′ must be higher than or equal to the priority of T″. Test cases T′ with failure records or not yet executed should be executed before any other test case T″. f is a priority function that assigns a priority value to test case T based on its execution status (pass / fail / not executed). The execution order T″ of all test cases in set P is represented by (T″∈P), where T″ is an element of set P. T″≠T′ represents comparing the execution orders of two different test cases when T″ is not equal to T′. [f(T′)≥f(T″)] represents determining the optimal priority order of candidate test case sets by comparing the values ​​of f for any two test case execution orders T′ and T″.

[0155] For example, when both test cases fail to execute, the execution order is further prioritized based on historical status. The correlation strength between the two test cases is compared, and the test case with a higher correlation strength with historical test cases is executed first. If the correlation strength is the same, the execution time of the test cases is compared, and the test case with the shorter execution time is executed first, thus determining the final test priority order.

[0156] In step 406, test cases are executed according to the execution order.

[0157] For example, based on the selected set of candidate test cases, the candidate test cases are executed in a determined execution order to realize the testing process of system functions. Within the same execution time, the purpose of the testing work can be made clearer, and the effectiveness of the testing can be improved.

[0158] In step 407, the test results are fed back to the historical dataset.

[0159] For example, after the candidate test cases are executed according to the execution order, the test results of the test cases are obtained, and the test results are fed back to the historical dataset to update the data in the historical dataset.

[0160] The test case processing method provided in this application has the following beneficial effects:

[0161] By reading test case data and filtering test cases according to target requirements, a minimum set of smoke test cases is generated. Then, based on historical test datasets, the priority ranking of the candidate test case set is determined. Prioritization based on historical data can identify which functions or modules have had problems in past versions, allowing for a focus on these high-risk areas, improving testing effectiveness, and effectively reducing the risk of releasing unstable versions. Test case prioritization based on historical data also supports rapid iteration and frequent releases, quickly verifying new features and changes in each iteration by optimizing the testing process. Adding weights based on test case execution status in the test case prioritization ensures that test cases are updated and replaced across different test versions. Within the same execution time, this makes the testing objective clearer and the selected test cases more efficient at fault detection.

[0162] The following description continues to illustrate the exemplary structure of the test case processing method apparatus 455 provided in the embodiments of this application as a software module. In some embodiments, such as... Figure 2 As shown, the software modules in the test case processing method apparatus 455 stored in the memory 450 may include: a test case extraction module 4551, used to obtain a set of candidate test cases and test requirement information, wherein the set of candidate test cases includes multiple candidate test cases; extracting target test cases that meet the test requirement information from the set of candidate test cases; and a test case sorting module 4552, used to determine the execution order of each target test case based on historical test data; and to execute the test process according to the execution order and each target test case.

[0163] In some embodiments, the test requirement information includes a change requirement set and a total requirement set, the total requirement set including multiple test requirements, and the change requirement set being a subset of the total requirement set; the test case extraction module 4551 is further configured to extract a first test case set that satisfies the test requirements in the change requirement set from the candidate test case set, wherein the first test case set includes candidate test cases that satisfy the test requirements in the change requirement set, and the first test case set is the smallest set that satisfies the change requirement set; subtracting the change requirement set from the total requirement set yields the first requirement set; selecting all candidate test cases that satisfy the first requirement set from the remaining candidate test cases after subtracting the first test case set from the candidate test case set and adding them to the first test case set to form a second test case set; and using the candidate test cases in the second test case set as target test cases.

[0164] In some embodiments, the test case extraction module 4551 is further configured to: determine, based on the first requirement set, the remaining requirement coverage of the candidate test cases remaining after subtracting the first test case set from the candidate test case set, wherein the remaining requirement coverage is the number of test requirements satisfied by the candidate test cases in the first requirement set; select the candidate test case corresponding to the largest remaining requirement coverage and add it to the first test case set to form an updated first test case set; and determine the second requirement set composed of the test requirements satisfied by the candidate test cases in the updated first test case set; when the second requirement set is different from the total requirement set, update the first requirement set according to the unselected candidate test cases, and perform the step of determining the remaining requirement coverage of the remaining candidate test cases in the candidate test case set based on the first requirement set; when the second requirement set is the same as the total requirement set, use the updated first test case set as the second test case set.

[0165] In some embodiments, historical test data includes: the execution status and test case parameters of each target test case; the test case sorting module 4552 is further configured to divide the target test cases into multiple test case groups according to the type of execution status corresponding to each target test case, wherein the target test cases in each test case group have the same execution status; determine the inter-group execution order of each test case group according to the priority of the execution status type; determine the intra-group execution order of the target test cases in each test case group according to the test case parameters; and sort the execution order between each target test case according to the inter-group execution order and the intra-group execution test cases.

[0166] In some embodiments, the test case sorting module 4552 is further configured to traverse each target test case and add the traversed target test cases to the test case group corresponding to the corresponding execution state type, wherein the execution state type includes: not executed, executed successfully, and executed failed.

[0167] In some embodiments, the test case parameters include: test case execution duration and requirement association value; the test case sorting module 4552 is further configured to perform the following processing for each test case group: sort each target test case in descending order according to the requirement association value of each target test case to obtain a first sort; adjust the order of multiple target test cases with the same requirement association value according to the test case execution duration to obtain the execution order within the group, wherein the adjustment is based on the ascending order of the test case execution duration.

[0168] This application provides a computer program product, which includes a computer program or computer-executable instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer-executable instructions from the computer-readable storage medium and executes the computer-executable instructions, causing the electronic device to perform the test case processing method described above in this application.

[0169] This application provides a computer-readable storage medium storing computer-executable instructions or a computer program. When the computer-executable instructions or the computer program are executed by a processor, the processor will execute the processing method of the test cases provided in this application, for example, such as... Figure 3A The test case processing method is shown.

[0170] In some embodiments, the computer-readable storage medium may be a memory such as RAM, ROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0171] In some embodiments, computer-executable instructions may take the form of programs, software, software modules, scripts, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as stand-alone programs or as modules, components, subroutines, or other units suitable for use in a computing environment.

[0172] As an example, computer-executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple co-located files (e.g., files that store one or more modules, subroutines, or code sections).

[0173] As an example, computer-executable instructions can be deployed to execute on a single electronic device, or on multiple electronic devices located at one location, or on multiple electronic devices distributed across multiple locations and interconnected via a communication network.

[0174] In summary, the embodiments of this application extract target test cases that meet the testing requirements from the candidate test case set, ensuring that test cases that meet the requirements are selected. The execution order of each target test case is determined based on historical test data. The test process is executed according to the execution order and each target test case, test resources are allocated reasonably, and tests are executed in a targeted manner according to the execution order, thereby improving the effectiveness and efficiency of testing.

[0175] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A method for processing test cases, characterized in that, The method includes: Obtain a set of candidate test cases and test requirement information, wherein the set of candidate test cases includes multiple candidate test cases; Extract target test cases that satisfy the test requirements from the set of candidate test cases; The execution order of each target test case is determined based on historical test data; The testing process is executed according to the execution order and each of the target test cases.

2. The method according to claim 1, characterized in that, The test requirement information includes a set of change requirements and a set of total requirements. The set of total requirements includes multiple test requirements, and the set of change requirements is a subset of the set of total requirements. The step of extracting target test cases that satisfy the test requirement information from the candidate test case set includes: Extract a first set of test cases that satisfy the test requirements in the change requirement set from the candidate test case set, wherein the first set of test cases includes candidate test cases that satisfy the test requirements in the change requirement set, and the first set of test cases is the smallest set that satisfies the change requirement set; Subtracting the total set of requirements from the set of changed requirements yields the first set of requirements; From the remaining candidate test cases obtained by subtracting the first test case set from the candidate test case set, select all candidate test cases that satisfy the first requirement set and add them to the first test case set to form the second test case set; The candidate test cases in the second set of test cases are used as the target test cases.

3. The method according to claim 2, characterized in that, The step of selecting all candidate test cases that satisfy the first requirement set from the remaining candidate test cases after subtracting the first test case set from the candidate test case set, and adding them to the first test case set to form the second test case set, includes: Based on the first set of requirements, determine the remaining requirement coverage of the candidate test cases remaining after subtracting the first set of test cases from the candidate test case set, wherein the remaining requirement coverage is the number of test requirements in the first set of requirements satisfied by the candidate test cases; The candidate test cases corresponding to the largest number of remaining requirements covered are selected and added to the first test case set to form an updated first test case set. The second requirement set is determined by the test requirements satisfied by the candidate test cases in the updated first test case set. When the second requirement set is different from the total requirement set, the first requirement set is updated according to the unselected candidate test cases, and the step of determining the remaining requirement coverage of the remaining candidate test cases in the candidate test case set based on the first requirement set is executed based on the updated first test case set. When the second set of requirements is the same as the total set of requirements, the updated first set of test cases is used as the second set of test cases.

4. The method according to claim 1, characterized in that, The historical test data includes: the execution status and test case parameters of each target test case; Determining the execution order of each target test case based on historical test data includes: Based on the type of execution state corresponding to each target test case, the target test cases are divided into multiple test case groups, wherein the execution state of the target test cases in each test case group is the same; The execution order between each test case group is determined based on the priority of the execution status type. The execution order of the target test cases within each test case group is determined based on the test case parameters. The execution order of each target test case is sorted according to the inter-group execution order and the intra-group execution test case order.

5. The method according to claim 4, characterized in that, The step of dividing the target test cases into multiple test case groups based on the type of execution state corresponding to each target test case includes: Each target test case is traversed, and the traversed target test cases are added to the test case group corresponding to the execution status type. The execution status types include: not executed, executed successfully, and executed failed.

6. The method according to claim 4, characterized in that, The test case parameters include: test case execution duration and requirement association values; The step of determining the execution order of target test cases within each test case group based on the test case parameters includes: For each of the aforementioned test case groups, the following processing is performed: Based on the requirement association value of each target test case, sort each target test case in descending order to obtain the first sort; The execution order of multiple target test cases with the same requirement association value is adjusted according to the execution duration of the test cases to obtain the execution order within the group, wherein the adjustment is based on the ascending order of the execution duration of the test cases.

7. A test case processing apparatus, characterized in that, The device includes: The test case extraction module is used to obtain a set of candidate test cases and test requirement information, wherein the set of candidate test cases includes multiple candidate test cases; and to extract target test cases that meet the test requirement information from the set of candidate test cases. The test case sorting module is used to determine the execution order of each target test case based on historical test data; and to execute the test process according to the execution order and each target test case.

8. An electronic device, characterized in that, The electronic device includes: Memory is used to store executable instructions or computer programs. A processor, configured to execute computer-executable instructions or computer programs stored in the memory, implements the processing method for test cases as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing computer-executable instructions or a computer program, characterized in that, When the computer-executable instructions or computer program are executed by a processor, they implement the test case processing method according to any one of claims 1 to 6.

10. A computer program product comprising computer-executable instructions or a computer program, characterized in that, When the computer-executable instructions or computer program are executed by a processor, they implement the processing method for the test cases according to any one of claims 1 to 6.

Images