Test case execution method, electronic equipment, storage medium and program product

Abstract

The invention discloses a test case execution method, electronic equipment, a storage medium and a program product, and relates to the technical field of software testing, and the method comprises the steps: obtaining test data features in a test case execution process, carrying out the grading of a test case based on the obtained test data features to determine a grade coefficient, according to the method and the device, the priority of the test case is dynamically adjusted according to the test data characteristics of the test case, the test case is executed according to the priority score of each test case, the execution sequence of the test case is dynamically adjusted, the dynamic adaptability of the test case is improved, and the test efficiency is improved. Therefore, the test efficiency can be ensured, and the determination of the priority score also fuses the test module and the test function point, so that the test cases of the important module and the core function point can have higher priority scores, the execution is preferentially performed, and the coverage integrity of the core function and the error-prone module can be ensured.

Description

Technical Field

[0001] This application relates to the technical field of software testing, and particularly to a method for executing test cases, an electronic device, a storage medium, and a program product. Background Art

[0002] With the exponential growth of the complexity of software systems, the scale and execution cost of test cases have increased sharply.

[0003] In related technologies, software testing methods rely on manual experience to design test cases and statically assign priorities to each test case, and each test case is executed in a fixed priority order. However, the fixed execution order results in a mixture of high-time-consuming test cases and low-risk test cases, which is likely to drag down the overall test efficiency. Summary of the Invention

[0004] This application provides a method for executing test cases, an electronic device, a storage medium, and a program product, so as to at least solve the problem of low test efficiency caused by the fixed use case execution order in related technologies.

[0005] This application provides a method for executing test cases, including: Obtaining a set of test cases; Executing the test cases in the set of test cases, and obtaining test data characteristics during the execution of the test cases; Based on the test function points of the test cases and the corresponding test data characteristics, determining the level coefficients corresponding to the test cases; Based on the test modules, test function points of the test cases, and the corresponding test data characteristics, determining the use case weights corresponding to the test cases; Based on the level coefficients and the use case weights corresponding to the test cases, determining the priority scores corresponding to the test cases in the subset of test cases; Executing the test cases in the subset of test cases according to the priority scores.

[0006] This application also provides an apparatus for executing test cases, including: A use case acquisition module, configured to obtain a set of test cases; A feature acquisition module, configured to execute the test cases in the set of test cases and obtain test data characteristics during the execution of the test cases; A coefficient determination module, configured to determine the level coefficients corresponding to the test cases based on the test function points of the test cases and the corresponding test data characteristics; A weight determination module, configured to determine the use case weights corresponding to the test cases based on the test modules, test function points of the test cases, and the corresponding test data characteristics; A priority determination module, configured to determine a priority score corresponding to a test case in the test case subset based on the grade coefficient corresponding to the test case and the use case weight; An execution module, configured to execute the test cases in the test case subset according to the priority score.

[0007] The present application further provides an electronic device, including: a memory, configured to store a computer program; a processor, configured to implement the steps of any one of the above-mentioned test case execution methods when executing the computer program.

[0008] The present application further provides a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by a processor, the steps of any one of the above-mentioned test case execution methods are implemented.

[0009] The present application further provides a computer program product, including a computer program, and when the computer program is executed by a processor, the steps of any one of the above-mentioned test case execution methods are implemented.

[0010] Through the present application, since test data characteristics are obtained during the execution of test cases, the test cases are classified according to the obtained test data characteristics to determine the grade coefficient, and then the corresponding priority score is determined, realizing the dynamic adjustment of the priority of test cases according to the test data characteristics of the test cases. Furthermore, the test cases are executed according to the priority scores of the test cases, realizing the dynamic adjustment of the execution order of the test cases, improving the dynamic adaptability of the test cases, thereby ensuring the test efficiency. Moreover, the determination of the priority score also integrates the test module and the test function point, which can ensure that the test cases of important modules and core function points have higher priority scores and are thus executed preferentially, contributing to ensuring the coverage integrity of the core functions and error-prone modules. Therefore, the technical problem of low test efficiency caused by using a fixed use case execution order can be solved, and the technical effect of improving the test efficiency can be achieved. Description of the Drawings

[0011] To more clearly illustrate the embodiments of the present application, the following will briefly introduce the drawings required in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0012] Figure 1 It is a schematic diagram of the execution architecture of test cases for an exemplary embodiment of the present application; Figure 2 It is a schematic flowchart of a test case execution method provided by an embodiment of the present application; Figure 3 A schematic flowchart of another method for executing a test case provided by an embodiment of the present application; Figure 4 A schematic flowchart of yet another method for executing a test case provided by an embodiment of the present application; Figure 5 A schematic flowchart of still another method for executing a test case provided by an embodiment of the present application; Figure 6 A schematic structural diagram of an apparatus for executing a test case provided by an embodiment of the present application. Detailed implementation manners

[0013] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0014] It should be noted that in the description of the present application, the terms "include", "comprise" or any other variant thereof are intended to cover a non-exclusive inclusion, such that a process, method, article or device including a series of elements includes not only those elements but also other elements not expressly listed, or further includes elements inherent to such process, method, article or device. The terms "first", "second", etc. in the present application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0015] With the exponential growth of software system complexity (such as distributed storage, cloud computing platforms, etc.), the scale and execution cost of test cases have increased sharply. In related static test methods, test cases are pre-designed relying on manual experience, and fixed execution priorities are statically assigned to test cases. For example, the execution priorities of test cases are set according to module importance, function point risks, etc., and the tests are executed according to the set execution priorities. However, when executing tests according to fixed priorities, it is impossible to dynamically adjust the case priorities based on real-time test results (such as defect distribution, environmental load). Especially in the storage field, for example, modules such as Network Attached Storage (NAS), active-active, and compression, with the explosive growth of function points and parameter combinations (such as cross-validation of Redundant Array of Independent Disks (RAID) levels, compression algorithms, cache policies, etc.), it is difficult for the fixed execution order to cope with challenges such as dynamic parameter combinations, environmental fluctuations, and uneven defect distributions, resulting in poor adaptability and low test efficiency. Moreover, the fixed execution order leads to the mixing of high-time-consuming cases and low-risk cases, which easily drags down the overall efficiency and causes resource waste.

[0016] In addition, the parameter combination coverage rate depends on manual enumeration, and it is difficult to exhaust the high-dimensional parameter space (such as the failure switching scenarios of the active-active module need to cover multi-dimensional parameters such as network latency, node status, and data consistency), resulting in insufficient parameter coverage.

[0017] To address the deficiencies brought by the fixed execution order, related technologies also provide a solution to dynamically adjust the case priorities through predefined rules (such as "if the number of defects > 3, then execute first"), but this solution has the problem of rigid rules. The rules rely on manual presetting and cannot adapt to complex scenarios (such as dynamically balancing the case execution time and stability when the resource load fluctuates).

[0018] In view of the above problems, the present application provides a method for executing test cases. This solution is based on dynamic data-driven multi-objective test case optimization and solves the above problems through the following improvement points: (1) Dynamic feature engineering. This solution collects multi-dimensional data such as test logs, defect reports, resource monitoring, and parameter coverage in real time, constructs a joint feature space covering case execution time, defect density, resource load, and historical failure rate, and provides real-time input for optimization; (2) Hierarchical rule and weight fusion. Business rules (such as core function mandatory testing, parameter combination constraints) are encoded into hierarchical strategies (the first layer is mandatory testing, the second layer is dynamic layer elevation), and combined with the multi-objective optimization of the NSGA-II algorithm to narrow the search space and accelerate convergence; (3) Improvement of NSGA-II with real-time feedback. Inject dynamic weights (such as increasing the risk objective weight when the defect density increases) into the crossover and mutation operations of the genetic algorithm to ensure that the optimization results adapt to environmental changes. This solution constrains the case priority through real-time hierarchical rules, fuses multi-feature weight calculations (defect density, resource load, etc.), and realizes the dynamic adaptability of test cases: responds to environmental changes at the minute level, automatically raises / lowers the case level and adjusts the execution order; achieves multi-objective balance based on the NSGA-II algorithm: generates an optimal solution set in terms of coverage, risk, and time consumption, and supports on-demand selection for business scenarios. Test results show that this solution significantly improves test efficiency (execution time reduced by 30%), coverage (increased by 18%), and defect detection rate (high-risk defect detection rate > 95%), and is applicable to the efficient verification of complex systems.

[0019] To enable those skilled in the art of this technical field to better understand the solution of the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0020] Combined with the specific application environment architecture or specific hardware architecture on which the execution of the method for executing test cases depends, the specific application environment architecture or specific hardware architecture is described herein.

[0021] Figure 1 It is a schematic diagram of the execution architecture of test cases for an exemplary embodiment of the present application. As Figure 1 shown, the overall architecture includes several parts: real-time data collection, historical data feedback, business strategy input, dynamic layering and weight calculation, and multi-objective optimization. The dynamic layering and weight calculation part dynamically adjusts the test case layering and calculates the priority weight according to the input business strategy, real-time data, and historical data; the multi-objective optimization part combines the NSGA-II algorithm and the dynamic layering and weight calculation to generate an optimal set of test cases; for the obtained set of test cases, the case priority is dynamically adjusted based on the test data features output by the dynamic layering and weight calculation part during the test process to achieve dynamic adjustment of the test case execution order.

[0022] An embodiment of the present application provides a method for executing test cases. The method will be described in detail in combination with the execution process of the method for executing test cases.

[0023] Figure 2 FIG. is a schematic flowchart of a method for executing test cases provided by an embodiment of the present application. This method can be executed by an execution device for test cases in an embodiment of the present application. The device can be implemented in software and / or hardware and can be integrated in an electronic device.

[0024] As Figure 2 shown, the method for executing test cases includes the following steps: Step 101, obtain a test case set.

[0025] Among them, the test case set includes multiple test cases.

[0026] Exemplarily, multiple test cases can be designed manually according to test requirements to obtain a test case set.

[0027] Exemplarily, based on preset constraint conditions, multiple test cases that meet the constraint conditions can be generated to obtain a test case set.

[0028] Exemplarily, an initial use case set can be obtained first, and then based on a preset objective function, the NSGA-II algorithm can be used for multi-objective optimization to obtain an optimal test case set. Among them, the initial use case set can be obtained based on historical test cases or designed manually. The present application does not limit this.

[0029] Step 102, execute the test cases in the test case set and obtain the test data characteristics during the execution of the test cases.

[0030] In this embodiment, for the obtained test case set, the test cases in the test case set can be executed, and the test data characteristics during the execution of the test cases can be obtained.

[0031] Among them, the test data characteristics are obtained by performing feature processing on the acquired data. Different types of data use different feature processing methods, and the feature processing methods for each type of data can be set according to actual needs. The present application does not limit this.

[0032] In the embodiments of the present application, the timing of obtaining data can be triggered by an event or collected periodically, and the present application does not limit this. Among them, event-triggered collection can include, for example: when a test case fails, automatically capture the current environment snapshot (such as logs, memory dumps) to obtain relevant data; when the parameter coverage rate of a certain module's function point is lower than a threshold (such as <80%), trigger data collection to enhance sampling. Periodic collection can include, for example: summarize data such as coverage rate, execution time, and defect distribution after each round of testing to achieve data collection; collect test data according to a preset collection period (such as 24 hours).

[0033] As an example, the collected test data can include but are not limited to defect data and environment data. Among them, defect data can be obtained by collecting the defect distribution during test execution in real time (such as defect type, triggering parameter combination, repair status, etc.); environment data can be obtained by obtaining the device information of the device under test, including but not limited to the proportion of active users in combinations such as device model, operating system version, CPU usage rate, memory usage rate, and network configuration.

[0034] For the data of each test case obtained, it can be characterized according to the corresponding feature processing method to obtain the test data features corresponding to each test case.

[0035] As an example, Table 1 shows an example table of feature classification in an exemplary embodiment of the present application. As shown in Table 1, the feature types can include execution features, defect features, coverage features, environment features, and timing features, and the feature names and calculation logics of each type of feature are given. Based on Table 1, the data of each test case obtained can be converted into the feature values of the corresponding features to obtain test data features.

[0036] Table 1

[0037] In an alternative embodiment of the present application, the obtained test data features can also be processed such as normalized and discretized to standardize the test data features and reduce the risk of inaccurate calculation of priority weights caused by large data differences. Among them, normalization can scale features with different dimensions (such as elapsed time, coverage rate) to the interval [0,1], and the normalized value of the feature = (feature value - feature minimum value) / (feature maximum value - feature minimum value). Discretization can divide continuous features (such as resource load) into different levels (such as low / medium / high load). In addition, joint features across modules - functions can also be constructed, such as the failure rate of the combination of "NAS - delete + high load".

[0038] For example, taking the feature generation of the "create" function of the NAS module as an example, assume that the acquired original data is: {"module": "NAS"; "function point": "create"; "time consumption": "150s"; "parameter combination": "RAID5+SSD"; "result": "failure"; "CPU load": "85%"}, then the processed features are shown in Table 2 below.

[0039] Table 2

[0040] In the embodiment of the present application, by collecting multi-source data such as test logs, defect reports, resource monitoring, and parameter coverage in real time, dynamically integrating according to event-triggered and stream-processing rules, data features covering execution efficiency, defect risk, and environmental load are constructed, providing data support for subsequent calculation of priority weights.

[0041] Step 103: Determine the grade coefficient corresponding to the test case based on the test function point of the test case and the corresponding test data features.

[0042] In this embodiment, after obtaining the test data features of the test case, the grade coefficient corresponding to the test case can be determined based on the test function point (abbreviated as function point) included in the test case and the corresponding test data features. Among them, the higher the grade coefficient, the higher the execution grade of the test case, and the greater the possibility of being executed preferentially.

[0043] As an example, the test function point of the test case and the corresponding test data features can be input into a pre-trained grade prediction model, and the grade prediction model makes a grade prediction according to the input data and outputs the corresponding grade coefficient.

[0044] Step 104: Determine the use case weight corresponding to the test case based on the test module, test function point, and corresponding test data features of the test case.

[0045] As an example, for each test case, the test module and function point of the test case can be obtained, and the test module, function point, and corresponding test data features are input into a pre-trained weight prediction model, and the predicted weight output by the weight prediction model is determined as the use case weight of the test case.

[0046] As an example, according to a preset multi-feature weight calculation algorithm, in combination with the test modules, test function points and corresponding test data features of test cases, the case weights corresponding to each test case can be determined. Among them, the multi-feature weight calculation algorithm can be preset. For example, the multi-feature weight calculation algorithm can include preset weight values corresponding to different modules and different function points, and include weight calculation formulas corresponding to different test data features. According to the test data features corresponding to a test case, the corresponding weight value can be calculated according to the weight calculation formula of the corresponding feature, and then the weight values of each feature are accumulated with the weight values corresponding to the test modules and test function points included in the test case to obtain the case weight corresponding to the test case.

[0047] Step 105: Based on the grade coefficient and case weight corresponding to the test case, determine the priority score corresponding to the test case in the test case subset.

[0048] In this embodiment, after determining the case weight and grade coefficient of the test case, the priority score corresponding to each test case can be determined based on the case weight and grade coefficient. For example, the product of the case weight and grade coefficient of a test case can be determined as the priority score of the test case, that is, priority score = case weight × grade coefficient. It should be noted that the higher the priority score, the higher the priority of the test case, the greater the probability of being executed first, and the earlier the execution order.

[0049] Step 106: Execute the test cases in the test case subset according to the priority score.

[0050] In this embodiment, after calculating the priority scores of each test case, the test cases in the test case subset can be executed according to the priority scores. For example, it can start from the test case subset with the highest execution level and execute the test cases in the test case subset in descending order of the priority scores. For another example, the test cases in the test case subsets of each execution level can be sorted in descending order of the priority scores, and a certain number of test cases with the highest ranking can be obtained from the test case subsets respectively and executed in parallel to execute multiple test cases simultaneously and improve the execution efficiency.

[0051] It should be noted that in the embodiment of the present application, the above process can be repeatedly executed, that is, in the process of executing test cases in multiple rounds, data is collected and test data features are determined through data collection trigger conditions, and then the execution level of the test cases is dynamically adjusted and new priority weights are determined, and the execution order of the test cases is adjusted according to the newly determined priority weights.

[0052] In the embodiments of the present application, since the priority score is determined by comprehensively considering test data such as the test module, function point, parameter coverage rate, and defects of the test case, the test case with the highest priority score can be understood as the test case that best meets the current device environment and test requirements. Therefore, by adjusting the execution order of the test cases according to the priority score and preferentially executing the test cases with high priority scores, the test efficiency can be ensured.

[0053] For the method of executing test cases in the embodiments of the present application, since the test data characteristics are obtained during the execution of the test cases, the test cases are classified according to the obtained test data characteristics to determine the level coefficient, and then the corresponding priority score is determined. This realizes the dynamic adjustment of the priority of the test cases according to the test data characteristics of the test cases. Furthermore, the test cases are executed according to the priority scores of each test case, realizing the dynamic adjustment of the execution order of the test cases, improving the dynamic adaptability of the test cases, and thus being able to ensure the test efficiency. Moreover, the determination of the priority score also incorporates the test module and test function point, which can ensure that the test cases of important modules and core function points have higher priority scores and are thus preferentially executed, helping to ensure the coverage integrity of the core functions and error-prone modules. Therefore, the technical problem of low test efficiency caused by using a fixed test case execution order can be solved, achieving the technical effect of improving the test efficiency.

[0054] In an alternative embodiment of the present application, as Figure 3 shown, on the basis of the foregoing embodiment, step 103 may include the following sub-steps: Step 201, classify the test cases in the test case set based on the test function point of the test case, the corresponding test data characteristics, and the execution classification rule, to obtain subsets of test cases with different execution levels.

[0055] It should be noted that in the embodiments of the present application, the execution classification rule can be designed according to actual needs, including the trigger conditions corresponding to different execution levels. When a certain trigger condition is met, the test case is classified into the corresponding execution level. For example, when the function points included in the test case are core functions such as creation and deletion, the test case is classified into the highest execution level (required test item); when the parameter combination coverage rate of the test case is lower than a certain value, the test case is classified into the second highest execution level; when the function point of the test case is a non-core function such as a query operation or a low-risk use case, the test case is classified into the lowest execution level.

[0056] In this embodiment, after obtaining the test data features of the test cases, the test function points of the test cases and the corresponding test data features can be matched with the trigger conditions corresponding to each execution level in the execution level division rule. For example, the matching can start from the highest execution level. If the trigger condition of the current execution level is matched, the test case is classified into the current execution level; otherwise, the trigger condition of the next execution level is continuously matched until an execution level is matched. After all the test cases are matched, the stratification of the test cases is completed, and subsets of test cases with different execution levels are obtained.

[0057] In an alternative embodiment of the present application, the execution level division rule includes at least one trigger condition corresponding to at least one execution level, and can classify multiple test cases into at least one execution level. When any one of the at least one trigger condition is satisfied, the test case is classified into the execution level corresponding to the satisfied trigger condition. That is to say, the execution level division rule includes at least one execution level, and one execution level corresponds to at least one trigger condition. If a test case matches any one of the trigger conditions corresponding to an execution level, the test case is classified into that execution level.

[0058] As an example, as shown in Table 3, the execution level division rule can include trigger conditions corresponding to three execution levels. The three execution levels are the first layer (required test items), the second layer, and the third layer (regular items), which are used to classify test cases into three levels. It can be seen from Table 3 that test cases with high defect density are classified into the highest execution level. Dynamic optimization combines historical defect data with real-time operation results, which helps to prioritize the execution of use cases that are prone to trigger defects (such as boundary values and abnormal inputs), thereby improving the defect discovery efficiency.

[0059] Table 3

[0060] In the embodiments of the present application, when classifying test cases by level, the test cases in the test case set can be traversed, and the function points of the current test case (for the convenience of description and distinction, referred to as target function points) and the test data characteristics (for the convenience of description and distinction, referred to as target test data characteristics) can be obtained. Then, starting from the highest execution level in the execution level classification rule (for example, the first layer in Table 3), a match is made. The target function points and target test data characteristics are matched with the target trigger conditions corresponding to the current execution level. If the target function points and / or target test data characteristics meet at least one of the target trigger conditions, the current test case is classified into the current execution level; if the target function points and target test data characteristics do not meet all the conditions in the target trigger conditions, continue to traverse to the next execution level, take the next execution level as the current execution level, and match the target function points and target test data characteristics with the corresponding trigger conditions. Repeat the above process until all the test cases in the test case set are traversed, and the test cases are classified into different execution levels, obtaining subsets of test cases at different execution levels.

[0061] For example, the "query" function of the NAS module is automatically promoted to the first layer due to a defect density of 0.25 and a historical failure rate of 0.35, and the execution level is increased. When the parameter coverage rate of the "modify" function of the compression module drops to 65%, it is promoted to the second layer.

[0062] It can be understood that in the embodiments of the present application, the execution levels of test cases support dynamic updates. When the promotion conditions are met, a promotion is triggered, that is, the execution level of the test case is increased; when the demotion conditions are met, a demotion is triggered, that is, the execution level of the test case is decreased. For example, the promotion conditions can include, but are not limited to, the defect density and / or historical failure rate exceeding the corresponding thresholds, and stability test cases triggered when the resource load is high (such as the delete operation of the dual-active module under high load); the demotion conditions can include, but are not limited to, the parameter coverage rate > 90% and the historical failure rate < 0.1, and automatically reducing the level of non-core use cases when the resource load is low.

[0063] In the embodiments of the present application, by traversing from the highest execution level in the execution level classification rule, the target function points and target test data characteristics of the current test case are matched with the target trigger conditions corresponding to the current execution level. If at least one of the target trigger conditions is matched, the current test case is classified into the current execution level. Thus, each test case can be classified into the highest-level execution level that it meets, ensuring the accuracy of the level classification and laying the foundation for subsequent accurate calculation of priority weights.

[0064] Step 202, query the mapping relationship table between the execution level and the level coefficient, and determine the level coefficients corresponding to the subsets of test cases at different execution levels. The level coefficient is positively correlated with the execution level.

[0065] Among them, the mapping relationship table between the execution level and the level coefficient can be preset according to actual needs. The level coefficients corresponding to different execution levels are different. This mapping relationship table records the level coefficients corresponding to different execution levels. Among them, the level coefficient is positively correlated with the execution level. That is to say, the higher the execution level, the larger the corresponding level coefficient.

[0066] In this embodiment, for each subset of test cases at each execution level, by querying the mapping relationship table between the execution level and the level coefficient, the level coefficient corresponding to each subset of test cases at each execution level can be determined, and thus the level coefficient corresponding to each test case is determined. As an example, taking the execution level including three levels: the first level, the second level, and the third level as an example, the level coefficients corresponding to each execution level are shown in Table 4.

[0067] Table 4

[0068] For each subset of test cases, according to its corresponding execution level, the above mapping relationship table can be queried to determine the corresponding level coefficient.

[0069] The execution method of the test cases in the embodiment of the present application first divides the test cases into subsets of test cases at different execution levels according to the execution level division rule, then queries the mapping relationship table between the execution level and the level coefficient to determine the level coefficients corresponding to the subsets of test cases at different execution levels, and further determines the priority score based on the level coefficient. Since the level coefficient is positively correlated with the execution level, it maximally ensures that the test cases with a higher execution level have a higher priority score, thereby ensuring that the test cases with a higher execution level are executed first and improving the execution rate of the test cases with a higher level.

[0070] In an alternative embodiment of the present application, as Figure 4 shown, on the basis of the foregoing embodiment, step 201 may include the following sub-steps: Step 301, obtain the test module and test function point of the test case.

[0071] In this embodiment, when determining the use case weight of any test case, the test module and test function point (abbreviated as function point) corresponding to the test case can be obtained first.

[0072] Step 302, determine the basic weight of the test case based on the test module and test function point.

[0073] As an example, when determining the basic weight of the test case, the mapping relationship table between the test module - function point and the weight set in advance can be queried, and the weight corresponding to the test module - function point is determined as the basic weight.

[0074] As an example, when determining the basic weight of a test case, the importance factor corresponding to the test module and the risk factor corresponding to the test function point can be obtained. Among them, the importance factors corresponding to different test modules can be preset according to the actual system architecture. For example, the importance factors of different test modules are set as follows: NAS - 0.25, dual - active - 0.20, compression - 0.15, etc.; the risk factors corresponding to different test function points can be preset according to the actual operation risk. For example, the risk factors of different test function points are set as follows: create - 0.3, delete - 0.25, modify - 0.2, query - 0.1, etc. Moreover, the module importance weight and the function point risk weight can be obtained. Among them, the values of the module importance weight and the function point risk weight can be preset according to actual needs. If more attention is paid to the module importance, a larger weight can be set for the module importance parameter, that is, the module importance weight is larger. It should be noted that in the embodiments of the present application, the module importance weight and the function point risk weight are not restricted by the sum of the two being 1. Then, based on the module importance weight and the function point risk weight, the importance factor and the risk factor are weighted to obtain the basic weight of the test case. That is to say, the basic weight = module importance weight × importance factor of the test module + function point risk weight × risk factor of the test function point.

[0075] In the embodiments of the present application, by obtaining the importance factor corresponding to the test module, the risk factor corresponding to the test function point, and obtaining the module importance weight and the function point risk weight, and then weighted to obtain the basic weight of the test case. Thus, the influence degree of the test module and the function point on the use case priority can be flexibly adjusted through the module importance weight and the function point risk weight, improving the flexibility of the method.

[0076] Step 303, based on the test data characteristics corresponding to the test case, determine the dynamic factor of the test case.

[0077] As an example, the test data characteristics can be input into a pre - trained dynamic factor prediction model. The dynamic factor prediction model makes a prediction based on the input test data characteristics and outputs the predicted dynamic factor as the dynamic factor corresponding to the test case.

[0078] As an example, based on a preset feature factor calculation rule, the feature factor corresponding to the test data characteristics can be determined, and then based on the feature factor, the dynamic factor of the test case can be determined. For example, the sum of the feature factors corresponding to the test data characteristics can be used as the dynamic factor of the test case, or the mean value of the feature factors corresponding to the test data characteristics can be used as the dynamic factor of the test case.

[0079] It should be noted that the calculation rules of the characteristic factors can be preset according to actual needs. They can be fixed values set in advance. For example, when the characteristic value of a certain characteristic is within a range, it corresponds to one value, and when it is within another range, it corresponds to another value. Or, they can also be corresponding calculation formulas. For each characteristic, there is a calculation formula for the characteristic factor, and the characteristic factors of each characteristic can be calculated using the calculation formula.

[0080] Exemplarily, Table 5 shows the characteristic factors corresponding to different characteristics and their calculation formulas. For example, the calculation formula corresponding to the defect density factor is 0.3×(number of defects / number of test cases). It can be understood that the coefficient 0.3 can be dynamically adjusted according to actual needs and cannot be regarded as a limitation to this application. Similarly, the coefficients in the calculation formulas of other characteristic factors can also be dynamically adjusted according to needs. Table 5 is only used as an example to explain this application and cannot be regarded as a limitation to this application.

[0081] Table 5

[0082] For example, assume the test case is: the "delete" function of the active-active module, and the basic weight = 0.20 (module importance) + 0.25 (function point risk) = 0.45. The characteristic factors determined according to Table 5 are as follows: Defect density = 0.3 (number of defects / number of test cases = 3 / 10), and the defect density factor is 0.3×0.3 = 0.09; Parameter coverage rate = 60%, and the coverage rate factor is 0.2×(1 - 0.6) = 0.08; Historical failure rate = 0.4, and the historical failure rate factor is 0.15×0.4 = 0.06; Resource load = 90%, and the resource load factor is 0.1×0.9 = 0.09; Assume the dynamic factor is the sum of each characteristic factor, then the dynamic factor of this test case is 0.09 + 0.08 + 0.06 + 0.09 = 0.32.

[0083] In the embodiments of this application, by determining the characteristic factors corresponding to the test data characteristics based on the preset calculation rules of the characteristic factors, and then determining the dynamic factor of the test case based on the characteristic factors, using the corresponding calculation rules for different characteristics to determine the corresponding characteristic factors helps to improve the accuracy of the calculation results, and ultimately improves the accuracy of the priority score. Moreover, calculating the characteristic factors based on the preset rules supports dynamically adjusting the calculation rules of the characteristic factors according to actual needs, improving the flexibility of the method.

[0084] Step 304: Determine the test case weight corresponding to the test case based on the basic weight and the dynamic factor of the test case.

[0085] In this embodiment, after determining the basic weight and dynamic factor of the test case, the case weight of the test case can be determined based on the basic weight and dynamic factor of the same test case.

[0086] As an example, the product of the basic weight and the dynamic factor can be used as the case weight of the test case.

[0087] As an example, the weighted sum of the basic weight and the dynamic factor can be used as the case weight of the test case.

[0088] In an alternative embodiment of the present application, the test data feature further includes the time consumed for executing the case, that is, the duration consumed for executing one round of test cases. In this embodiment, the average execution time of the test cases in the test case set can be obtained. The average execution time is the average value of the execution times of all test cases in the test case set, and the time penalty factor corresponding to the test case can be determined based on the execution time corresponding to the test case and the average execution time. For example, the time penalty factor = w × (execution time / average execution time), where the value of w can be set according to actual needs. For example, w = 0.05 is set. Furthermore, when determining the case weight of the test case, the time penalty factor, the basic weight, and the dynamic factor can be combined for determination. Based on the basic weight, the dynamic factor, and the time penalty factor of the same test case, the case weight corresponding to the test case is determined.

[0089] As an example, the case weight = w1 × basic weight + w2 × dynamic factor - time penalty factor, where the values of w1 and w2 can be set according to actual needs.

[0090] As an example, the case weight = basic weight × (1 + dynamic factor) - time penalty factor.

[0091] Continuing with the above example, assuming that the execution time of the above test case is 200s and the average execution time is 150s, and the time penalty factor = 0.05 × (execution time / average execution time), then the time penalty factor of this test case is 0.05 × (200 / 150) = 0.067. Assuming that the case weight = basic weight × (1 + dynamic factor) - time penalty factor, then the case weight of this test case is 0.45 × (1 + 0.32) - 0.067 = 0.527.

[0092] In the embodiment of the present application, by obtaining the time consumed for executing a test case and determining the time penalty factor of the test case based on the time consumed for executing the case, and using the time penalty factor for calculating the case weight, thus, the calculation of the case weight takes into account the factor of the time consumed for the case, reduces the case weight of the test case with a long execution time, and thereby reduces the priority score of the test case with a long execution time, so that when other conditions are the same, the test case with a short execution time is preferentially executed, which is beneficial to improving the test coverage rate.

[0093] The method for executing a test case in the embodiment of the present application determines the basic weight and the dynamic factor of the test case respectively for the test module, the test function point and the test data feature, and finally determines the case weight. Thus, fusing data in different dimensions to determine the case weight of the test case helps to improve the accuracy of the case weight, and further improves the accuracy of the priority score of the test case.

[0094] For example, the parameter coverage rate of the "modification" function of the compression module is low (60%), the historical failure rate is 0.25, the defect density is 0.1, the resource load is 70%, the time consumed for executing the case is 180 s, and the average execution time is 150 s. Based on the execution level division rule defined in Table 3, the parameter coverage rate triggers a layer-up to the second layer. Based on the preset importance factor of the test module and the risk factor of the function point, it is determined that the basic weight of this test case is 0.15 (module importance) + 0.2 (risk factor of the function point) = 0.35; based on the calculation methods of different feature factors defined in Table 5, the feature factors of each feature are determined as: the defect density factor is 0.03, the coverage rate factor is 0.08, the historical failure rate factor is 0.0375, and the resource load factor is 0.07. Thus, the dynamic factor of this test case is determined to be 0.03 + 0.08 + 0.0375 + 0.07 = 0.2175; the time penalty factor is 0.05×(180 / 150) = 0.06, then the case weight is 0.35×(1 + 0.2175) - 0.06 = 0.366. Based on Table 4, the level coefficient of the execution level to which this test case belongs is determined to be 1.2. Thus, the priority score of this test case is 1.2×0.366 = 0.439. The priority score of this test case is relatively high and it is preferentially scheduled, so that the parameter coverage rate is increased to 85%. After preferentially executing this test case, 2 new defects are added, triggering the defect density to be updated to 0.15, and the case weight of this test case is further increased.

[0095] In an alternative embodiment of the present application, when executing the test cases in the test case subset according to the priority score, the test cases can be executed in combination with the current resource load of the device. Thus, when executing the test cases, the current resource load can be obtained, and when the current resource load is within the preset load range, the test cases in the test case subset are executed according to the priority score.

[0096] Among them, the preset load range can be preset according to actual requirements. For example, the preset load range can be set to [50%-80%]. When the resource load is not less than 50% and not higher than 80%, it is determined that the resource load is within the preset load range.

[0097] In this embodiment, before executing the test cases, the current resource load of the device can be obtained first. The current resource load can be determined according to the current CPU usage rate and / or the current memory usage rate of the device. For example, the weighted sum of the current CPU usage rate and the current memory usage rate can be determined as the current resource load; alternatively, the current CPU usage rate or the current memory usage rate can be determined as the current resource load; or, the larger value of the current CPU usage rate and the current memory usage rate can be determined as the current resource load. After determining the current resource load, it is determined whether the current resource load is within the preset load range. If it is within the preset load range, the test cases in the test case subset are executed according to the priority score.

[0098] Further, in an alternative embodiment of the present application, when the current resource load is greater than the upper limit value of the preset load range, it is detected whether there are preset test cases including preset function points in the test cases. Among them, the preset function points are pre-defined lightweight function points, and the predicted test cases are lightweight test cases including the preset function points. These test cases consume less resources when executed. For example, the preset function point is a query operation; if there are preset test cases, the preset test cases are executed according to the priority score of the preset test cases. Optionally, only the preset test cases can be executed, and the priority score of this part of the test cases is re-determined according to the execution results of the preset test cases, merged and compared with the priority scores of the other test cases not executed in the previous round, and the current resource load is re-obtained. Then, the corresponding execution strategy is selected according to the current resource load; it is also possible to execute the remaining test cases in descending order of the priority score after executing the preset test cases. If there are no preset test cases, the test cases with shorter execution time and / or smaller resource load can be screened out from the test cases and executed first to reduce the resource occupation while ensuring the coverage as much as possible.

[0099] In this embodiment, when the current resource load is greater than the upper limit value of the preset load range, it is first detected whether there is a preset test case including a preset function point in the test cases, and when there is a preset test case, the preset test case is executed according to the priority score of the preset test case. Thus, when the resource load of the device is relatively high, the preset test cases including the preset function point are preferentially executed, and the preset test cases are executed according to the priority score, which can avoid resource contention and excessive resource load of the device while ensuring that the target test cases are executed according to the priority score, and improve the test coverage as much as possible.

[0100] In an alternative embodiment of the present application, when the current resource load is less than the lower limit value of the preset load range, the execution duration of the test cases is further obtained, and the test cases are sorted in descending order according to the execution duration, and a preset number of target test cases ranked at the front are determined, and then the target test cases are executed in parallel. After the target test cases are executed, a preset number of test cases can be determined from the remaining test cases as new target test cases according to the execution duration, and the newly determined target test cases are executed in parallel until all test cases are executed and this round of testing ends. Alternatively, after the target test cases are executed, the remaining test cases can be executed in turn according to the priority scores of the remaining test cases.

[0101] Among them, the value of the preset number can be preset according to actual needs. For example, the preset number is set to 5.

[0102] In this embodiment, when the current resource load of the device is less than the lower limit value of the preset load range, a preset number of test cases with a relatively high execution duration are preferentially obtained, and these test cases are executed in parallel. It can be understood that the test cases with a longer execution duration are usually complex cases, such as the full-link test cases of the dual-active module. Therefore, when the current resource load is relatively small, executing the test cases with a long execution duration in parallel can shorten the overall test cycle and improve the test efficiency.

[0103] In the embodiment of the present application, by obtaining the current resource load of the device and selecting a corresponding strategy to execute the test cases according to the current resource load, the dynamic execution optimization of the test cases combines the resource load situation (such as server performance, test environment status), automatically adjusts the case execution order and concurrency strategy based on the resource load, can shorten the overall test cycle, and improve the resource allocation efficiency.

[0104] To ensure the parameter coverage rate, defect detection rate, etc. of test cases, the test case set obtained in the embodiments of the present application can be an optimal test case solution set generated by fusing business rule constraints (such as mandatory coverage of core functions) and dynamic weights (parameter coverage rate, risk, time consumption) based on the NSGA-II algorithm through non-dominated sorting and crowding distance calculation. Thus, in an alternative embodiment of the present application, as Figure 5 shown, based on the foregoing embodiment, step 101 may include the following sub-steps: Step 401, obtain an initial test case set with a target quantity.

[0105] Among them, the value of the target quantity can be preset according to actual needs.

[0106] As an example, the initial test case set can be the historical test cases with the target quantity obtained.

[0107] As an example, the initial test case set can be test cases randomly generated that meet preset constraint conditions. For example, function points and parameter combinations can be randomly selected for each test module to generate test cases, thereby obtaining the initial test case set, where the generated test cases meet the constraint conditions. The constraint conditions can be predefined according to actual needs. For example, the constraint conditions are: (1) each test module covers at least 1 function point (core function verification); (2) the total test time ≤ 4 hours; (3) parameter combination validity (such as for the "delete" operation, the corresponding resource needs to exist first, that is, there needs to be a "create" case before "delete"). When obtaining the initial test case set, test cases with the target quantity that meet the constraint conditions can be randomly generated to obtain the initial test case set; or, multiple test cases that meet the constraint conditions can also be randomly generated, and then test cases with the target quantity are selected from them to form the initial test case set. For example, the generated test cases can be stratified based on the execution level division rule, and the test cases in the first layer are preferentially selected, followed by the second layer.

[0108] For example, the initial test case set can be simply represented as shown in Table 6 below. Each test case is defined as a vector, expressed as [module type, function point, parameter combination, weight]. In Table 6, the weight corresponding to the test case refers to the basic weight in the foregoing embodiment, which is determined according to the module importance of the test module and the risk factor of the function point in the test case.

[0109] Table 6

[0110] Step 402, based on a multi-objective function, determine multiple objective values of the initial test cases in the initial test case set.

[0111] Among them, the multi-objective function includes multiple objective functions, and the objective functions can be defined according to actual test requirements. This application places no restrictions on the multi-objective function.

[0112] As an example, the multi-objective function includes a parameter coverage rate function, a defect risk function, and a test execution time function. Among them, the parameter coverage rate function is expressed as: parameter coverage rate = number of covered parameters / total number of parameters; the defect risk function is expressed as: risk score = defect density × historical failure rate × resource load factor; the test execution time function is expressed as: total time consumption = ∑(test case time consumption × execution frequency). During the iterative optimization process of test cases, the optimization objectives are to maximize the parameter coverage rate, minimize the defect risk, and minimize the test execution time. By defining the above multi-objective function, using the defined multi-objective function as the optimization objective during the test case iteration process, the finally obtained optimal test cases after iterative optimization of the test cases satisfy the above multi-objective function. Therefore, using the selected test cases for testing can improve the parameter coverage rate and defect detection rate of the test, and reduce the test time consumption, thereby improving the test efficiency.

[0113] In this embodiment, for the obtained initial use case set, multiple objective values of each initial use case in the initial use case set can be determined based on the pre-defined multi-objective function.

[0114] Step 403: Perform non-dominated sorting and crowding distance calculation on the initial use cases based on the multiple objective values to obtain multiple non-dominated use case layers and the crowding distance corresponding to the initial use cases.

[0115] In this embodiment, based on the multiple objective values corresponding to each initial use case, the initial use cases can be sorted non-dominantly to obtain multiple non-dominated use case layers, and the crowding distance of each initial use case can be calculated.

[0116] Taking the multiple objective values including parameter coverage rate (denoted as Coverage), execution time consumption (denoted as Time), and defect risk (denoted as Risk) as an example, the condition for use case A to dominate use case B is: Coverage A ≥Coverage B and Time A ≤Time B and Risk A ≥Risk B . Based on this domination relation determination condition, the initial use cases can be sorted non-dominantly.

[0117] For example, assume there are three initial use cases: Use Case 1 (Coverage = 60%, Time = 10 min, Risk = 0.16), Use Case 2 (Coverage = 50%, Time = 15 min, Risk = 0.10), and Use Case 3 (Coverage = 60%, Time = 20 min, Risk = 0.18). Through the above domination relation determination conditions, it can be determined that Use Case 1 dominates Use Case 2, and Use Case 1 and Use Case 3 do not dominate each other.

[0118] After determining the non - domination relationships among the initial use cases and obtaining multiple non - dominated use case layers, the crowding distance can be further calculated. The larger the crowding distance, the sparser the distribution of use cases, and the greater the probability of a use case being retained. When calculating the crowding distance, the crowding distance is calculated for the initial use cases in the same non - dominated level. The crowding distance calculation formula is as follows: 。

[0119] In the above formula, crowd( i ) represents the crowding distance of the i th initial use case, f j ( i + 1) represents the objective value of the ( i + 1)th initial use case on the j th objective function, f jmax represents the maximum value of the objective values on the j th objective function, and f jmin represents the minimum value of the objective values on the j th objective function.

[0120] Through the above crowding distance calculation formula, the crowding distance of each initial use case in each non - dominated use case layer can be determined. For example, if the difference in the Time dimension between adjacent use cases is large (such as 5 minutes and 20 minutes), the crowding distance is higher. By calculating the crowding distance, high - risk but time - consuming use cases may be retained due to sparsity.

[0121] Step 404: Based on multiple non - dominated use case layers and the crowding distances corresponding to the initial use cases, select the target number of initial use cases to form the next - generation use case set.

[0122] In this embodiment, after determining multiple non - dominated use case layers of the initial use cases and the crowding distances of the initial use cases, based on multiple non - dominated use case layers and the crowding distances corresponding to the initial use cases, select the target number of initial use cases to form the next - generation use case set.

[0123] Among them, when selecting initial use cases to form the next-generation use case set, the initial use cases in the highest layer of multiple non-dominated use case layers are preferentially selected, that is, the non-dominated use cases determined first are selected. If these use cases are less than the target quantity, continue to obtain from the next lower non-dominated use case layer. If the quantity exceeds the target quantity after adding the use cases in the next lower non-dominated use case layer, then sort the use cases in this layer according to the crowding distance, and start selecting from the use case with the largest crowding distance until the target quantity of use cases is obtained to form the next-generation use case set.

[0124] Step 405, perform genetic operations on the next-generation use case set to obtain a set of offspring use cases.

[0125] Among them, the genetic operations can include crossover and / or mutation. Crossover means exchanging the test modules or function points of two parent use cases, and mutation means randomly adjusting the parameter combination or weight.

[0126] In this embodiment, after obtaining the next-generation use case set, genetic operations can be performed on the test cases therein to obtain a set of offspring use cases. Among them, the number of offspring use cases included in the set of offspring use cases can be the same as the number of test cases included in the next-generation use case set, which is also the target quantity.

[0127] For example, to obtain offspring use cases through the crossover method, Parent 1 is ["NAS", "Create", "Parameter A", 0.8], and Parent 2 is ["Active-Active", "Delete", "Parameter B", 0.7], then the offspring use case obtained by crossover is ["NAS", "Delete", "Parameter B", 0.75] (mixing modules and function points).

[0128] Another example, to obtain offspring use cases through mutation, the original use case is ["Compress", "Query", "Parameter C", 0.6], and the mutated use case is ["Compress", "Query", "Parameter D", 0.6] (change in parameter combination).

[0129] Step 406, use the combined use case set obtained by combining the next-generation use case set and the set of offspring use cases as the initial use case set for iteration until the iteration termination condition is met, and determine the test case set based on the obtained next-generation use case set.

[0130] In this embodiment, the obtained set of offspring use cases is combined with the next-generation use case set to obtain a combined use case set, and the combined use case set is used as the initial use case set for iteration. Iteratively execute steps 402 to 406, that is, perform the calculation of the target value, non-dominated sorting, and crowding distance calculation on the newly obtained initial use case set again, select the next-generation use case set and perform genetic operations to obtain a set of offspring use cases, and repeat the above process iteratively until the iteration termination condition is met, and determine the test case set based on the obtained next-generation use case set.

[0131] It should be noted that the iteration termination condition can be preset. For example, the iteration termination condition is set to include at least one of the number of iterations reaching a preset value (for example, 50 times), the parameter coverage rate reaching a preset threshold (for example, 95%), and the iteration duration reaching a preset duration.

[0132] In this embodiment, when determining the test case set based on the obtained next-generation use case set, the non-dominated use cases in the next-generation use case set can be determined as the test case set, or the next-generation use case set can be determined as the test case set. This application does not limit this.

[0133] The execution method of the test cases in the embodiments of this application sets multiple objective functions, determines the objective values of the initial use cases based on the multiple objective functions, performs non-dominated sorting and crowding distance calculation on the initial use cases based on the objective values, selects the next-generation use cases based on the non-dominated sorting results and crowding distances and performs genetic operations to obtain the offspring use case set, and repeats the iteration to obtain the test case set. Thus, an optimal use case set that satisfies the multiple objective functions can be obtained for testing, balancing the coverage rate and execution efficiency, and providing support for improving the parameter coverage rate, defect detection rate, and test efficiency.

[0134] Furthermore, in an alternative embodiment of this application, in the real-time testing stage, the fitness of each test case can also be determined based on the multiple objective functions, and the priorities of the test cases can be adjusted based on the fitness. The test cases with higher fitness are executed first. For example, after executing the test cases in the test case subset according to the priority scores, in the subsequent testing process, the test results (such as defect density, resource load, historical failure rate, parameter coverage rate, execution time) are obtained after each round of testing, and the objective values of each test case are calculated based on the above multiple objective functions, and the weight coefficients of each objective value are updated based on the test results. The corresponding fitness is calculated based on the objective values and the new weight coefficients, and the test cases are executed in the order from high to low according to the new fitness.

[0135] Among them, the calculation method of the fitness is as follows: 。

[0136] In the above formula, α 、 and γ are weight coefficients, and their values can be dynamically adjusted according to the test results in the real-time testing stage. The adjustment range each time can be set according to actual needs. For example, if during the testing process, the defect density of a certain test case rises from 0.2 to 0.3, its risk score weight coefficient ( )automatically decreases, for example, from β =0.4 to β =0.35, to reduce the fitness of this test case, thereby reducing its execution priority.

[0137] In this embodiment, the priority (fitness) of test cases is dynamically adjusted through indicators such as defect density and historical failure rate, so that the executed test cases preferentially cover high-risk modules, ensuring the coverage integrity of core functions and error-prone modules, and strengthening the coverage of high-risk areas.

[0138] Optionally, in the real-time testing stage, the test cases can also be eliminated and optimized through a preset elimination strategy. For example, test cases that have not been selected for N consecutive rounds and have a weight lower than the threshold are moved to the retention pool and eliminated after a delay of M cycles. The values of N and M can be set according to actual requirements.

[0139] Through the description of the above implementation manners, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases, the former is a better implementation manner.

[0140] The embodiment of the present application also provides an execution device for test cases. This device can be implemented by software and / or hardware and can be integrated in an electronic device.

[0141] Figure 6 For a structural schematic diagram of an execution device for test cases provided by an embodiment of the present application, as Figure 6 shown, the execution device 50 for test cases includes: a test case acquisition module 510, a feature acquisition module 520, a coefficient determination module 530, a weight determination module 540, a priority determination module 550, and an execution module 560.

[0142] Among them, the test case acquisition module 510 is used to acquire a test case set; The feature acquisition module 520 is used to execute the test cases in the test case set and acquire the test data features during the execution of the test cases; The coefficient determination module 530 is used to determine the grade coefficient corresponding to the test case based on the test function point of the test case and the corresponding test data features; The weight determination module 540 is used to determine the case weight corresponding to the test case based on the test module, test function point of the test case, and the corresponding test data features; The priority determination module 550 is used to determine the priority score corresponding to the test cases in the test case subset based on the grade coefficient and case weight corresponding to the test case; The execution module 560 is used to execute the test cases in the test case subset according to the priority score.

[0143] Optionally, the coefficient determination module 530 includes: A grading unit, which is used to classify test cases in a test case set based on test function points of test cases, corresponding test data characteristics, and execution level division rules, so as to obtain subsets of test cases with different execution levels; A coefficient determination unit, which is used to query a mapping relationship table between execution levels and level coefficients, and determine the level coefficients corresponding to subsets of test cases with different execution levels, where the level coefficients are positively correlated with the execution levels.

[0144] Optionally, the execution level division rule includes at least one trigger condition corresponding to at least one execution level. When any one of the at least one trigger conditions is satisfied, the test case is classified into the execution level corresponding to the satisfied trigger condition; The grading unit is further used to: traverse the test cases in the test case set, and obtain the target function point and target test data characteristics of the current test case; start matching from the highest execution level in the execution level division rule, and match the target function point and target test data characteristics with the target trigger condition corresponding to the current execution level; in the case where the target function point and / or target test data characteristics satisfy at least one of the target trigger conditions, classify the current test case into the current execution level; after traversing all the test cases in the test case set, obtain subsets of test cases with different execution levels.

[0145] Optionally, the weight determination module 540 includes: An acquisition unit, which is used to acquire the test module and test function point of a test case; A first determination unit, which is used to determine the basic weight of a test case based on the test module and test function point; A second determination unit, which is used to determine the dynamic factor of a test case based on the test data characteristics corresponding to the test case; A third determination unit, which is used to determine the use case weight corresponding to a test case based on the basic weight and dynamic factor of the test case.

[0146] Optionally, the first determination unit is further used to: acquire the importance factor corresponding to the test module and the risk factor corresponding to the test function point; acquire the module importance weight and the function point risk weight; based on the module importance weight and the function point risk weight, weight the importance factor and the risk factor to obtain the basic weight of the test case.

[0147] Optionally, the second determination unit is further used to: determine the characteristic factor corresponding to the test data characteristics based on a preset characteristic factor calculation rule; determine the dynamic factor of the test case based on the characteristic factor.

[0148] Optionally, the test data characteristics further include the time consumed for executing the use case, and the use case weight determination sub-module further includes: An average time-consuming acquisition unit for acquiring the average execution time-consuming of test cases in a test case set; A time penalty determination unit for determining a time penalty factor corresponding to a test case based on the execution time-consuming of the execution case corresponding to the test case and the average execution time-consuming; A third determination unit is further configured to: determine a use case weight corresponding to a test case based on the basic weight, dynamic factor, and time penalty factor of the test case.

[0149] Optionally, the execution module 560 is further configured to: acquire the current resource load; and execute the test cases in the test case subset according to the priority score when the current resource load is within a preset load range.

[0150] Optionally, the execution module 560 is further configured to: detect whether there is a preset test case including a preset function point in the test cases when the current resource load is greater than the upper limit value of the preset load range; if there is a preset test case, execute the preset test case according to the priority score of the preset test case.

[0151] Optionally, the execution module 560 is further configured to: acquire the execution time duration of the test cases when the current resource load is less than the lower limit value of the preset load range; sort the test cases in descending order of the execution time duration, determine a preset number of target test cases ranked at the front; and execute the target test cases in parallel.

[0152] Optionally, the use case acquisition module 510 is further configured to: acquire an initial use case set with a target number; determine multiple target values of the initial use cases in the initial use case set based on a multi-objective function; perform non-dominated sorting and crowding distance calculation on the initial use cases based on the multiple target values to obtain multiple non-dominated use case layers and the crowding distance corresponding to the initial use cases; select the initial use cases with the target number from the multiple non-dominated use case layers and the crowding distance corresponding to the initial use cases to form a next-generation use case set; perform genetic operations on the next-generation use case set to obtain a child use case set; and iterate the combined use case set obtained by combining the next-generation use case set and the child use case set as the initial use case set until the iteration termination condition is satisfied, and determine the test case set based on the obtained next-generation use case set.

[0153] Optionally, the multi-objective function includes a parameter coverage rate function, a defect risk function, and a test execution time function; in the iterative optimization process of the test cases, the optimization objectives are to maximize the parameter coverage rate, minimize the defect risk, and minimize the test execution time.

[0154] For the description of the features in the embodiments corresponding to the execution device of the test cases, reference may be made to the relevant descriptions in the embodiments corresponding to the execution method of the test cases, which will not be elaborated here one by one.

[0155] An embodiment of the present application further provides an electronic device, including a memory and a processor. A computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in the embodiment of the execution method of any of the above test cases.

[0156] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored. The computer program is configured to execute the steps in the embodiment of the execution method of any of the above test cases when running.

[0157] In an exemplary embodiment, the above computer-readable storage medium may include, but is not limited to: various media that can store computer programs such as USB flash drives, read-only memories (ROM for short), random access memories (RAM for short), mobile hard disks, magnetic disks, or optical discs.

[0158] An embodiment of the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, it implements the steps in the embodiment of the execution method of any of the above test cases.

[0159] An embodiment of the present application further provides another computer program product, including a non-volatile computer-readable storage medium. The non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements the steps in the embodiment of the execution method of any of the above test cases.

[0160] Those skilled in the art can further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Skilled professionals can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0161] The above has introduced in detail an execution method of a test case, an electronic device, a storage medium, and a program product provided by the present application. Specific examples are used in this article to elaborate on the principle and implementation manner of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application. It should be noted that for those of ordinary skill in the art in the technical field, without departing from the principle of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the protection scope of the claims of the present application.

Claims

1. A method for executing a test case, characterized in that: include: Get a collection of test cases; Executing the test cases in the test case set and obtaining test data features during the execution of the test cases; Determining a grade coefficient corresponding to the test case based on the test function points of the test case and the corresponding test data characteristics; Determine a test case weight corresponding to the test case based on the test module, test function point and corresponding test data characteristics of the test case; Determine, based on the grade coefficient corresponding to the test case and the case weight, the priority score corresponding to the test case in the test case set; The test cases in the test case set are executed according to the priority scores.

2. The test case execution method according to claim 1, characterized in that: The step of determining the grade coefficient corresponding to the test case based on the test function points of the test case and the corresponding test data features includes: Based on the test function points of the test cases and the corresponding test data features and execution level classification rules, the test cases in the test case set are classified into different levels to obtain a subset of test cases with different execution levels; The execution level and level coefficient mapping relationship table is queried to determine the level coefficient corresponding to the subset of test cases of different execution levels, wherein the level coefficient is positively correlated with the execution level.

3. The test case execution method according to claim 2, characterized in that: The execution level classification rule includes at least one trigger condition corresponding to at least one execution level, and when any one of the at least one trigger condition is met, the test case is classified into the execution level corresponding to the satisfied trigger condition; The test cases in the test case set are graded based on the test function points of the test case and the corresponding test data features and the execution grade classification rules to obtain a subset of test cases with different execution grades, including: Traversing the test cases in the test case set, and obtaining the target function points and target test data features of the current test case; Starting from the highest execution level in the execution level classification rule, matching the target function point and the target test data feature with the target trigger condition corresponding to the current execution level; When the target function point and / or the target test data feature meets at least one of the target trigger conditions, classifying the current test case into the current execution level; After traversing all the test cases in the test case set, a subset of test cases at different execution levels is obtained.

4. The method for executing a test case according to claim 1, characterized in that: The determining of the test case weight corresponding to the test case based on the test module, the test function point and the corresponding test data feature of the test case includes: Obtaining the test modules and test function points of the test case; Determining a basic weight of the test case based on the test module and the test function point; Determining a dynamic factor of the test case based on test data characteristics corresponding to the test case; Based on the basic weight of the test case and the dynamic factor, a case weight corresponding to the test case is determined.

5. The test case execution method according to claim 4, characterized in that: The determining the basic weight of the test case based on the test module and the test function point includes: Obtaining the importance factor corresponding to the test module and the risk factor corresponding to the test function point; Obtain module importance weights and function point risk weights; Based on the module importance weight and the function point risk weight, the importance factor and the risk factor are weighted to obtain the basic weight of the test case.

6. The method for executing a test case according to claim 4, characterized in that: The determining the dynamic factor of the test case based on the test data feature corresponding to the test case includes: Determine the characteristic factor corresponding to the test data feature based on a preset characteristic factor calculation rule; Based on the characteristic factor, a dynamic factor of the test case is determined.

7. The test case execution method according to claim 4, characterized in that: The test data feature also includes the time taken to execute the test case, and the method further includes: Obtaining the average execution time of the test cases in the test case set; Determine a time penalty factor corresponding to the test case based on the execution time corresponding to the test case and the average execution time; The determining, based on the basic weight of the test case and the dynamic factor, a use case weight corresponding to the test case includes: Based on the basic weight of the test case, the dynamic factor and the time penalty factor, a case weight corresponding to the test case is determined.

8. The test case execution method according to claim 1, characterized in that: The executing the test cases in the test case set according to the priority scores includes: Get the current resource load; When the current resource load is within a preset load range, the test cases in the test case set are executed according to the priority scores.

9. The test case execution method according to claim 8, characterized in that: The method further comprises: In a case where the current resource load is greater than an upper limit value of the preset load range, detecting whether there is a preset test case containing a preset function point in the test case; If the preset test case exists, the preset test case is executed according to the priority score of the preset test case.

10. The test case execution method according to claim 8, characterized in that: The method further comprises: When the current resource load is less than the lower limit of the preset load range, obtaining the execution time of the test case; Sorting the test cases in descending order of the execution time, and determining a preset number of target test cases that are ranked first; The target test cases are executed in parallel.

11. The method for executing a test case according to any one of claims 1 to 10, characterized in that: The obtaining of the test case set includes: Get the target number of initial use case sets; Based on the multi-objective function, determining multiple objective values ​​of the initial use cases in the initial use case set; Based on the multiple target values, non-dominated sorting and crowding distance calculation are performed on the initial use cases to obtain multiple non-dominated use case layers and crowding distances corresponding to the initial use cases; Based on the crowding distances corresponding to the multiple non-dominated use case layers and the initial use cases, selecting the target number of initial use cases to form a next-generation use case set; Performing genetic operations on the next-generation use case set to obtain a descendant use case set; The combined use case set obtained by merging the next generation use case set and the child use case set is used as the initial use case set for iteration until an iteration termination condition is met, and the test case set is determined based on the obtained next generation use case set.

12. The test case execution method according to claim 11, characterized in that: The multi-objective function includes a parameter coverage function, a defect risk function and a test execution time function; In the iterative optimization process of test cases, the optimization goals are to maximize parameter coverage, minimize defect risks and minimize test execution time.

13. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the method for executing a test case as claimed in any one of claims 1 to 12 when executing the computer program.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the method for executing a test case according to any one of claims 1 to 12.

15. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method for executing the test case according to any one of claims 1 to 12 are implemented.

Images