Software testing method, system and device

Abstract

The present application discloses a software testing method, system, and device, which relate to the field of software testing technology, including: executing each test file in a test file set in a current order in a target test environment; generating environmental impact information based on target execution information of the test file set; adjusting the execution order of each test file in the current order based on the environmental impact information to obtain a target order, wherein the target order is used as the order for the next execution of the test file set, and when the impact of executing the first test file on the target test environment indicated by the environmental impact information is greater than the impact of executing the second test file on the target test environment indicated by the environmental impact information, the execution order of the second test file in the target order precedes the execution order of the first test file. Through this application, the technical problem of low software testing efficiency is solved, and the technical effect of improving software testing efficiency is achieved.

Description

Technical Field

[0001] The present application relates to the field of software testing technology, and in particular to a software testing method, system, and device. Background Art

[0002] Software testing plays a crucial role in ensuring software quality, reducing software costs, and improving user satisfaction. Currently, the testing approach involves executing the entire set of test files corresponding to the software within a single test environment. However, the execution order of each test file within the set is random, which can easily lead to a situation where some test files damage the test environment, resulting in a lack of sufficient test environment for subsequent test files to execute. This, in turn, results in low software testing efficiency. Summary of the Invention

[0003] The present application provides a software testing method, system and device to at least solve the problem of low software testing efficiency in related technologies.

[0004] The present application provides a software testing method, comprising: executing each test file in a test file set in a target test environment in a current order, wherein the test file set is used to test software functions of the target software;

[0005] Generate environmental impact information based on target execution information of the test file set, wherein the target execution information is used to indicate the execution process and execution results of the test file set, and the environmental impact information is used to indicate the impact of executing each test file in the test file set on the target test environment;

[0006] The execution order of each test file in the current order is adjusted according to the environmental impact information to obtain a target order, wherein the target order is used as the order of executing the test file set next time. When the impact of executing the first test file on the target test environment indicated by the environmental impact information is greater than the impact of executing the second test file on the target test environment indicated by the environmental impact information, the execution order of the second test file in the target order precedes the execution order of the first test file.

[0007] The present application provides a software testing system, which includes: an execution machine, a scheduler and a listener, wherein:

[0008] an execution machine, configured to execute each test file in a test file set in a current order in a test environment under the control of the scheduler, wherein the test file set is used to test software functions of the target software;

[0009] A listener is used to obtain target execution information of the test file set and send it to the scheduler, wherein the target execution information is used to indicate the execution process and execution result of the test file set;

[0010] A scheduler is used to generate environmental impact information based on target execution information, wherein the environmental impact information is used to indicate the impact of executing each test file in the test file set on the test environment; adjust the execution order of each test file in the current order according to the environmental impact information to obtain a target order, wherein the target order is used as the order for executing the test file set next time. When the degree of impact of executing the first test file on the test environment indicated by the environmental impact information is greater than the degree of impact of executing the second test file on the test environment indicated by the environmental impact information, the execution order of the second test file in the target order precedes the execution order of the first test file.

[0011] The present application also provides an electronic device, comprising: a memory for storing a computer program; and a processor for implementing the steps of any of the above-mentioned software testing methods when executing the computer program.

[0012] The present application also provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned software testing methods are implemented.

[0013] The present application also provides a computer program product, comprising a computer program, which implements the steps of any of the above-mentioned software testing methods when executed by a processor.

[0014] Through the present application, each test file in the test file set is executed in the target test environment in the current order, and environmental impact information is generated according to the target execution information of the test file set to determine the impact of executing each test file in the test file set on the target test environment. Then, the execution order of each test file in the current order is adjusted according to the environmental impact information to obtain the order for executing the test file set for the next time. When the impact of executing the first test file on the target test environment indicated by the environmental impact information is greater than the impact of executing the second test file on the target test environment indicated by the environmental impact information, the execution order of the second test file in the target order precedes the execution order of the first test file, and the second test file with lower impact on the test environment is executed first, and then the first test file with higher impact on the environment is executed, so that the test environment can be used more and the waiting time for the execution of the test file to the test environment is reduced. Therefore, the technical problem of low software testing efficiency can be solved, and the technical effect of improving software testing efficiency can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0016] Figure 1 This is a hardware structure diagram of the software testing method of the embodiment of the present application;

[0017] Figure 2 is a flowchart of a software testing method according to an embodiment of the present application;

[0018] Figure 3 is a schematic diagram of a software testing system according to an embodiment of the present application;

[0019] Figure 4 is an architectural diagram of an intelligent testing method based on environmental self-healing and dynamic priority according to an embodiment of the present application;

[0020] Figure 5 This is a structural block diagram of a software testing device according to an embodiment of the present application. DETAILED DESCRIPTION

[0021] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0022] It should be noted that, in the description of this application, the terms "comprises," "includes," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. The terms "first," "second," etc., in this application are used to distinguish similar objects, and are not used to describe a particular order or sequence.

[0023] In order to enable those skilled in the art to better understand the present application, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0024] In conjunction with the specific application environment architecture or specific hardware architecture on which the execution of the software testing method depends, the specific application environment architecture or specific hardware architecture is described here.

[0025] The method embodiments provided in the embodiments of the present application can be executed in a server device or a similar computing device. Taking running on a server device as an example, Figure 1 This is a hardware structure diagram of the software testing method of the embodiment of the present application. Figure 1 As shown, the server device may include one or more ( Figure 1 Only one is shown) a processor 102 (the processor 102 may include but is not limited to a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data. The server device may also include a transmission device 106 and an input / output device 108 for communication functions. It will be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above server device. Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown.

[0026] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the software testing method in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, implementing the above-mentioned method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely located relative to the processor 102, and these remote memories can be connected to the server device via a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0027] Transmission device 106 is used to receive or transmit data via a network. A specific example of the aforementioned network may include a wireless network provided by a communication provider of the server device. In one embodiment, transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0028] The embodiments of the present application provide a software testing method, and the method is described in detail in conjunction with the execution flow of the software testing method.

[0029] In this embodiment, a software testing method is provided. Figure 2 is a flowchart of a software testing method according to an embodiment of the present application, such as Figure 2 As shown, the method includes the following steps:

[0030] Step S202: executing each test file in the test file set in the target test environment according to the current order, wherein the test file set is used to test the software functions of the target software;

[0031] Step S204: generating environmental impact information based on the target execution information of the test file set, wherein the target execution information is used to indicate the execution process and execution results of the test file set, and the environmental impact information is used to indicate the impact of executing each test file in the test file set on the target test environment;

[0032] Step S206: Adjust the execution order of each test file in the current order according to the environmental impact information to obtain a target order, wherein the target order is used as the order of executing the test file set next time. When the impact of executing the first test file on the target test environment indicated by the environmental impact information is greater than the impact of executing the second test file on the target test environment indicated by the environmental impact information, the execution order of the second test file in the target order precedes the execution order of the first test file.

[0033] Through the above steps, each test file in the test file set is executed in the target test environment in the current order, and environmental impact information is generated according to the target execution information of the test file set to determine the impact of executing each test file in the test file set on the target test environment. Then, the execution order of each test file in the current order is adjusted according to the environmental impact information to obtain the order for executing the test file set for the next time. When the impact of executing the first test file on the target test environment indicated by the environmental impact information is greater than the impact of executing the second test file on the target test environment indicated by the environmental impact information, the execution order of the second test file in the target order precedes the execution order of the first test file, and the second test file with lower impact on the test environment is executed first, and then the first test file with higher impact on the environment is executed, so that the test environment can be used more and the waiting time for the execution of the test file to the test environment is reduced. Therefore, the technical problem of low software testing efficiency can be solved, and the technical effect of improving software testing efficiency can be achieved.

[0034] In the embodiment provided in step S202, the test environment may refer to, but is not limited to, a collection of hardware, software, network and other resources built for software testing, and may include, but is not limited to: hardware environment, software environment and network environment, wherein the hardware environment may include, but is not limited to, computer hardware (including servers, personal computers, etc.) and peripheral hardware (including printers, scanners, cameras, etc.), the software environment may include, but is not limited to, operating systems, database systems and middleware, etc., and the network environment may include, but is not limited to, network topology (i.e., the connection method of computers, devices, etc. in the network) and network parameters (including bandwidth, delay, packet loss rate, etc.), etc.

[0035] Optionally, in this embodiment, the target test environment may be, but is not limited to, a test environment that meets the test environment requirements for executing the test file set.

[0036] Optionally, in this embodiment, the test file set may be, but is not limited to, designed for the target software, and the test file set may be, but is not limited to, used to test software functions of the target software.

[0037] Optionally, in this embodiment, the test file set may, but is not limited to, need to be executed multiple times in a loop in the target test environment to fully test the software functions of the target software.

[0038] Optionally, in this embodiment, during the first execution of the test file set, the current order may be, but is not limited to, a random order; during the non-first execution of the test file set, the current order may be, but is not limited to, an order determined based on the historical execution of the test file set.

[0039] Optionally, in this embodiment, the test file set may include, but is not limited to, multiple test files. These test files are independent of each other, and the execution order of these test files does not affect the execution results of the test files.

[0040] In the embodiment provided in step S204, the target execution information may be, but is not limited to, used to indicate the execution process and execution results of the test file set. The target execution information may be, but is not limited to, including the execution start time, execution end time, execution results, and resource usage during the execution process of each test file in the test file set (which may include, but is not limited to, CPU (Central Processing Unit) usage, memory usage, disk I / O (Input / Output), and network traffic), etc.

[0041] Optionally, in this embodiment, the environmental impact information may be used, but is not limited to, to indicate the impact of executing each test file in the test file set on the target test environment. The impact of executing each test file on the target test environment may include, but is not limited to, causing some test environments in the target test environment to be abnormally occupied, resulting in an inability to properly release the occupied environment resources after the test file is executed, resulting in a reduction in the number of test environments available later.

[0042] Optionally, in this embodiment, the environmental impact information may include, but is not limited to, an impact indicator and a no-impact indicator. The impact indicator indicates that executing the corresponding test file will have an impact on the target test environment, and the no-impact indicator indicates that executing the corresponding test file will not have an impact on the target test environment. The environmental impact information may also include, but is not limited to, an environmental impact level. The environmental impact level indicates the level of impact of executing the corresponding test file on the target test environment. The higher the environmental impact level, the greater the impact of executing the corresponding test file on the target test environment.

[0043] Optionally, in this embodiment, generating environmental impact information based on the target execution information of the test file set may include, but is not limited to, analyzing resource usage during the execution of the test file set to evaluate the impact of the test files on the target test environment. Specifically, during the execution of each test file, data such as CPU usage, memory usage, disk I / O, and network traffic is collected in real time; the collected data is analyzed to calculate statistical indicators such as averages and peak values; and the impact of the test files on the environment is evaluated based on the statistical indicators. For example, if the CPU usage of a test file continuously exceeds 80% during execution, it can be considered that the test file has a significant impact on the environment's CPU resources. For example, for test files A and B, during the execution of test file A, the CPU utilization rate reaches 90%, the memory utilization reaches 4 GB (Gigabyte), the disk I / O is frequent, and the network traffic is 10 MB / s (Megabytes per second). During the execution of test file B, the CPU utilization rate is 30%, the memory utilization is 1 GB, the disk I / O is less, and the network traffic is 1 MB / s. It can be seen that test file A has a greater impact on the CPU and memory resources of the target test environment, which may cause the system to run slowly. Test file B has less impact on the environment, and the system runs more smoothly. Ultimately, the environmental impact level of test file A is higher than that of test file B.

[0044] In the embodiment provided in step S206, the execution order of test files that have a greater impact on the target test environment after execution can be deployed after the execution order of test files that have a smaller impact on the target test environment after execution based on the environmental impact information, but is not limited to, to ensure that abnormal occupancy of the target test environment occurs in a later time period during the test process, thereby improving the utilization rate of the target test environment.

[0045] Optionally, in this embodiment, adjusting the execution order of each test file in the current sequence based on the environmental impact information may include, but is not limited to, selecting test files from the test set that will have an impact on the target test environment upon execution, adjusting the execution order of the selected test files to the end of the current sequence to obtain the target sequence; or selecting test files from the test set that will have an impact on the target test environment upon execution, sorting the selected test files according to their environmental impact levels, with higher environmental impact levels being ranked toward the end, to obtain a test file adjustment sequence, and adjusting the execution order of the test file adjustment sequence to the end of the current sequence to obtain the target sequence. For example, for a test file set including four test files, the current order of the four test files is test file A - test file B - test file C - test file D. When the current environmental impact information is used to indicate that the execution of test file B and test file C will affect the target test environment, the current order is adjusted to test file A-test file D-test file B-test file C; when the current environmental impact information is used to indicate that the execution of test file B and test file C will affect the target test environment and the environmental impact level of test file B is higher than the environmental impact level of test file C, the test file adjustment sequence is test file C-test file B, and finally the current order is adjusted to test file A-test file D-test file C-test file B.

[0046] Optionally, in this embodiment, during the execution of a test file collection, an interface provided by TestNG (Test Next Generation, an open source automated testing framework) can be monitored using IMethodInterceptor, which handles exceptions in the test file release environment and performs intelligent recovery based on the state of the environment before the test file execution. By performing intelligent self-healing during abnormal test file release, test case failures due to environmental interference can be reduced, thereby improving the test case pass rate.

[0047] As an optional implementation, environmental impact information is generated based on the target execution information of a test file set, including: detecting the test parameters of each test file to obtain multiple test parameters, wherein the test parameters are the ratio of the number of test cases that failed to execute in the test file to the total number of test cases included in the test file, and the target execution information includes the test parameters; and generating environmental impact information for each test file based on each test parameter.

[0048] Optionally, in this embodiment, each test file may include, but is not limited to, one or more test cases. When a test file includes multiple test cases, there is an association relationship between the test cases. Therefore, the execution order of the multiple test cases in the test file is fixed.

[0049] Optionally, in this embodiment, the test parameter may be, but is not limited to, the ratio of the number of test cases that failed to execute in the test file to the total number of test cases included in the test file. For example, when test file A includes b test cases, and there are a test cases that failed to execute during a certain execution of test file A, then the test parameter of test file A is a / b.

[0050] Optionally, in this embodiment, test cases that fail to execute are prone to abnormally occupy test resources of the target test environment. Environmental impact information of each test file can be generated based on, but is not limited to, various test parameters. The larger the test parameters of a test file, the greater the risk of affecting the target test environment.

[0051] Through the above content, the impact of each test file on the target test environment is judged by the execution of the test cases of each test file, and the impact risk of the target test environment is determined simply and feasible, providing strong support for the reasonable adjustment of the execution order of the test files.

[0052] As an optional implementation, environmental impact information of each test file is generated based on each test parameter, including: when a target test parameter among multiple test parameters is greater than or equal to a first threshold, generating an impact identifier of the target test file corresponding to the target test parameter, wherein the impact identifier is used to indicate that executing the target test file will have an impact on the target test environment; determining the impact identifier and the continuous failure parameter as environmental impact information, wherein the continuous failure parameter is the maximum number of continuous failures in the target test file, and the number of continuous failures is the number of consecutive failures of each test case that failed to execute in the target test file in the historical time period before the current moment, and the target execution information also includes the continuous failure parameter.

[0053] Optionally, in this embodiment, it is possible, but not limited to, when the target test parameter s / n (where n is the total number of test cases included in the test file and s is the number of test cases that failed to execute) is greater than or equal to the first threshold m, to determine that executing the target test file corresponding to the target test parameter will have an impact on the target test environment, generate an impact identifier, and determine the impact identifier and the continuous failure parameter as environmental impact information.

[0054] As an optional implementation, the execution order of each test file in the current order is adjusted according to the environmental impact information to obtain the target order, including: when the environmental impact information includes an impact identifier, calculating the product of the current priority parameter and the adjustment parameter to obtain the growth parameter of the priority parameter, wherein the adjustment parameter is the quotient of the continuous failure parameter divided by the reduction coefficient, and the current order is generated according to the current priority parameter of each test file; calculating the sum of the growth parameter and the current priority parameter to obtain the target priority parameter; generating the target order according to each target priority parameter of each test file in the test file set, wherein the test file with a larger priority parameter has a later execution order in the target order.

[0055] Optionally, in this embodiment, the target priority parameter P_new=P_old+(P_old×(x / 3)) can be, but is not limited to, P_old is the current priority parameter, x / 3 is the adjustment parameter, x is the continuous failure parameter, and 3 is the reduction coefficient.

[0056] Optionally, in this embodiment, each test file may have, but is not limited to, an initial default priority parameter, and the default priority parameter may be set to, but is not limited to, 5.

[0057] Optionally, in this embodiment, the adjustment range of the priority parameter of each test file may be, but is not limited to, limited. For example, the adjusted priority parameter may be, but is not limited to, less than or equal to the priority parameter threshold, and may be, but is not limited to, setting the priority parameter threshold to 8.

[0058] Through the above content, the execution order of each test file in the test file set is adjusted, so that the test files that are likely to affect the target test environment can be executed at a later position, reducing the abnormal time occupied by the target test environment and improving the testing efficiency of the software.

[0059] As an optional implementation, the following parameters can be defined, but are not limited to: P: the priority parameter of the test file (default value is 5), s: the number of use cases that failed to execute in the test file, n: the total number of use cases in the test file (s<=n), x: the number of consecutive failures (i.e., the consecutive failure parameter), and m: the failure rate threshold (default value is 0.4). If s / n>=m, the test file priority parameter adjustment is triggered. Specifically, the adjustment method is as follows: the adjusted priority parameter is always greater than the default value, and the cumulative priority increment of the same test file does not exceed β (the upper limit value can be, but is not limited to, set to 3) (i.e., the priority parameter threshold). The adjusted target priority parameter P_new = P_old + (P_old×(x / 3)), where P_old is the current priority parameter, x / 3 is the adjustment parameter, x is the consecutive failure parameter, and 3 is the reduction factor.

[0060] As an optional implementation, the default priority parameter (i.e., BASE_PRIORITY) of the test file can be set to 5, the threshold value y (i.e., THRESHOLD) of the continuous failure parameter can be set to 3, the highest priority parameter (i.e., MAX_PRIORITY) of the test file can be set to 10, and the amplification parameter α (i.e., COEFFICIENT) can be set to 2. When the continuous failure parameter of the test file exceeds the threshold value of the continuous failure parameter, the priority parameter is adjusted. The adjusted target priority parameter P_new = P_old + α(xy), where P_old is the current priority parameter. It should be noted that it is also necessary to ensure that the adjusted target priority parameter does not exceed the highest priority parameter of the test file. The following is a pseudocode implementation of the aforementioned method:

[0061] public class PriorityAdjuster implements IAnnotationTransformer {

[0062] private int calculateNewPriority(int x) {

[0063] final int BASE_PRIORITY = 5;

[0064] final int THRESHOLD = 3;

[0065] final int MAX_PRIORITY = 10;

[0066] final int COEFFICIENT = 2;

[0067] int delta = (x - PriorityConfig.FAILURE_THRESHOLD)

[0068] × PriorityConfig.PRIORITY_INCREMENT_COEFF;

[0069] return Math.min(

[0070] PriorityConfig.BASE_PRIORITY + delta,

[0071] PriorityConfig.MAX_PRIORITY );

[0073] }

[0074] }

[0075] As an optional implementation, environmental impact information of each test file is generated based on each test parameter, including: when a reference test parameter among multiple test parameters is greater than a first threshold and a continuous failure parameter is greater than or equal to a second threshold, an impact identifier of the reference test file corresponding to the reference test parameter is generated, wherein the impact identifier is used to indicate that executing the reference test file will have an impact on the target test environment, the continuous failure parameter is the maximum number of consecutive failures in the reference test file, the number of consecutive failures is the number of consecutive failures of each test case that failed to execute in the reference test file in the historical time period before the current moment, and the target execution information also includes the continuous failure parameter; determining the environmental impact information includes the impact identifier and the reference test parameter.

[0076] Optionally, in this embodiment, it is possible, but not limited to, when the reference test parameter is greater than a first threshold and the continuous failure parameter is greater than or equal to a second threshold, to determine that executing the reference test file corresponding to the reference test parameter will have an impact on the target test environment, generate an impact identifier, and determine the impact identifier and the reference test parameter as environmental impact information.

[0077] Optionally, in this embodiment, the first threshold may be set to, but is not limited to, 0.4.

[0078] Optionally, in this embodiment, the second threshold may be set to 3 but is not limited to it.

[0079] As an optional implementation, the execution order of each test file in the current order is adjusted according to the environmental impact information to obtain the target order, including: when the environmental impact information includes an impact identifier, calculating the product of the reference test parameter and the amplification factor to obtain the growth parameter of the priority parameter; calculating the sum of the growth parameter and the current priority parameter to obtain the target priority parameter, wherein the current order is generated according to the current priority parameter of each test file; generating the target order according to each target priority parameter of each test file in the test file set, wherein the test file with a larger priority parameter has a later execution order in the target order.

[0080] Optionally, in this embodiment, each test file may have, but is not limited to, an initial default priority parameter, and the default priority parameter may be set to, but is not limited to, 5.

[0081] Optionally, in this embodiment, the adjustment range of the priority parameter of each test file may be, but is not limited to, limited. For example, the adjusted priority parameter may be, but is not limited to, less than or equal to the priority parameter threshold, and may be, but is not limited to, setting the priority parameter threshold to 8.

[0082] Optionally, in this embodiment, the amplification factor can be set to, but is not limited to, 2.5.

[0083] Optionally, in this embodiment, the generation of the aforementioned target priority parameter may be achieved through, but not limited to, the following pseudo code:

[0084] / **

[0085] * Implement the IAlterSuiteListener interface for dynamic priority adjustment

[0086] * /

[0087] public class DynamicPriorityInjector implements IAlterSuiteListener {

[0088] private static final int X_THRESHOLD = 3; / / Continuous failure threshold (i.e. second threshold)

[0089] private static final double M_RATIO = 0.4; / / Failure rate threshold (ie, the first threshold)

[0090] private static final int P_DEFAULT = 5; / / Default priority parameter

[0091] @Override

[0092] public void alter(List <xmlsuite>suites) {

[0093] Map<String, TestClassStats> stats = loadHistoryStats(); / / Load historical execution records

[0094] suites.forEach(suite -> {

[0095] suite.getTests().forEach(test -> {

[0096] test.getClasses().forEach(clazz -> {

[0097] XmlClass xmlClass = (XmlClass) clazz;

[0098] String className = xmlClass.getName();

[0099] if (stats.containsKey(className)) {

[0100] TestClassStats s = stats.get(className);

[0101] / / Conditional judgment: failed x times in a row and failure rate > m

[0102] if (s.consecutiveFails >= X_THRESHOLD

[0103] && (double)s.failedCases.size() / s.totalCases > M_RATIO) {

[0104] / / Calculate the new priority (i.e. calculate the target priority parameter)

[0105] int delta = (int)((s.failedCases.size() / (double)s.totalCases) × 2.5);

[0106] int newPriority = Math.min(P_DEFAULT +delta, P_DEFAULT + 3); / / Upper limit control

[0107] xmlClass.getMethods().forEach(m ->m.setPriority(newPriority));

[0108] }

[0109] }

[0110] });

[0111] });

[0112] });

[0113] }

[0114] / / Historical statistical data structure

[0115] private static class TestClassStats {

[0116] int consecutiveFails; / / consecutive failures

[0117] Set <string>failedCases = new HashSet<>(); / / Failed case collection

[0118] int totalCases; / / Total number of use cases

[0119] }

[0120] }

[0121] Through the above content, the execution order of each test file in the test file set is adjusted, so that the test files that are likely to affect the target test environment are executed at a later position, reducing the abnormal occupancy time of the target test environment and improving the efficiency of using the target test environment.

[0122] As an optional implementation, the sum of the growth parameter and the current priority parameter is calculated to obtain the target priority parameter, including: calculating the sum of the growth parameter and the current priority parameter to obtain a preliminary priority parameter; when the preliminary priority parameter is greater than the priority parameter threshold of the test file set, determining the priority parameter threshold as the target priority parameter; when the preliminary priority parameter is less than or equal to the priority parameter threshold, determining the preliminary priority parameter as the target priority parameter.

[0123] Optionally, in this embodiment, an upper limit for adjusting the priority parameter of the test file may be set, but is not limited to the upper limit, and the priority parameter cannot exceed a priority parameter threshold.

[0124] Optionally, in this embodiment, alarm information for the test file can be generated, but is not limited to, when the preliminary priority parameter is greater than the priority parameter threshold of the test file set, wherein the alarm information is used to indicate that the degree of impact of the test file on the target test environment is abnormal, and the maintenance personnel will further analyze the cause of the abnormality of the test file after receiving the alarm information.

[0125] Through the above content, on the basis of adjusting the priority parameter to adjust the execution order of the test files, an upper limit for adjusting the priority parameter is also set, thereby avoiding meaningless adjustment of the priority parameter when the priority parameter is too high.

[0126] As an optional implementation, executing each test file in the test file set in the target test environment in the current order includes: matching the corresponding target sub-environment for each test file from each sub-environment based on the environmental information of each sub-environment and the test requirements of each test file, wherein the environmental information is used to indicate the test conditions possessed by the sub-environment, the test requirements are used to indicate the test conditions required to execute the test file, and the target test environment includes each sub-environment; executing each test file in the target sub-environment corresponding to each test file in the current order.

[0127] Optionally, in an embodiment of the present application, the target test environment may include but is not limited to multiple sub-environments, each sub-environment may integrate but is not limited to different test conditions, and may match corresponding target sub-environments for each test file from each sub-environment according to the test requirements of each test file, thereby realizing the simultaneous execution of different test files in the test file set on multiple sub-environments, thereby improving the testing efficiency of the software.

[0128] As an optional implementation, in the process of executing each test file in the test file set in the target test environment in the current order, the method also includes: detecting reference execution information of each test file, wherein the reference execution information is used to indicate the execution results of each test case in the test file; in the case where it is detected that the reference execution information of the alternative test file is used to indicate that the third test case in the alternative test file has failed to execute, detecting the retry requirement of the alternative test file according to the test failure information of the third test case, wherein the test failure information is used to indicate that the third test case has failed to execute in the target number of executions before the current execution, and the retry requirement is the requirement for the alternative test file to be executed again; in the case where the retry requirement is that the alternative test file needs to be executed again, executing the alternative test file in the backup test environment.

[0129] Optionally, in this embodiment, it is possible, but not limited to, when the third test case in the alternative test file fails to execute multiple times in a row, the retry requirement of the alternative test file can be detected based on the test failure information of the third test case, and when the alternative test file needs to be executed again, the alternative test file can be executed in the alternative test environment.

[0130] Optionally, in this embodiment, the backup test environment may be, but is not limited to, a test environment having the same test conditions as the target test environment.

[0131] Through the above content, when a test case fails continuously, it is determined whether the test case needs to be retried. If retry is required, the alternative test file is executed in the alternative test environment to avoid the impact of the previously executed test file on the test environment on the execution result of the alternative test file, and obtain more accurate execution process and execution results of the alternative test file.

[0132] As an optional implementation, detecting the retry requirement of the alternative test file based on the test failure information of the third test case includes: when the number of test failures of the third test case is greater than the failure number threshold, calculating the confidence parameter of the third test case by the following formula:

[0133] ,

[0134] Among them, the confidence parameter X is used to indicate the randomness of the failure of the execution of the third test case, n is the target number, f is the number of test failures, the number of test failures is the number of times the third test case failed in the target number of executions before the current execution, the test failure information includes the number of test failures, and C (n, k) is the number of combinations; when the confidence parameter is greater than the confidence parameter threshold, the retry requirement is determined to be that the alternative test file needs to be executed again.

[0135] Optionally, in this embodiment, it is possible, but not limited to, to determine that the alternative test file needs to be re-executed in a clean, unaffected alternative test environment when the number of test failures of the third test case is greater than the failure number threshold and the confidence parameter of the third test case is greater than the confidence parameter threshold.

[0136] Optionally, in this embodiment, the number of combinations C(n,k), also known as the binomial coefficient, represents the number of possible combinations of k elements from n different elements. Combinations here refer to selections that are made regardless of order. For example, C(5,2)=10 means there are 10 possible ways to select two items from five different items.

[0137] Optionally, in this embodiment, n may be set to, but not limited to, 5, the failure count threshold m (i.e., DEFAULT_M) may be set to 2, and the confidence parameter threshold (i.e., CONFIDENCE_THRESHOLD) may be set to 0.6. The aforementioned method may be implemented by, but not limited to, the following pseudo code:

[0138] public class SmartRetryAnalyzer implements IRetryAnalyzer {

[0139] private static final int DEFAULT_M = 2;

[0140] private static final double CONFIDENCE_THRESHOLD = 0.6;

[0141] private ClassHistory classHistory;

[0142] @Override

[0143] public boolean retry(ITestResult result) {

[0144] / / 1. Get historical data

[0145] int n = classHistory.getRecentExecutionCount();

[0146] int f = classHistory.getRecentFailures();

[0147] / / 2. Basic condition judgment

[0148] if (n < 5 || f <= DEFAULT_M) return false;

[0149] / / 3. Calculate confidence

[0150] double confidence = ConfidenceCalculator.calculateConfidence(n, f);

[0151] / / 4. Dynamic decision making

[0152] return confidence > CONFIDENCE_THRESHOLD;

[0153] }

[0154] / / Dynamic parameter configuration method

[0155] public void configure(

[0156] @Optional("5") int historySize,

[0157] @Optional("2") int failureThreshold,

[0158] @Optional("0.6") double confidenceThreshold) {

[0159] / / Configuration parameter logic

[0160] }

[0161] }

[0162] Through this application, a solution is proposed to solve the problem that probabilistic failure cases are difficult to reproduce during the execution of large-scale use cases. Through intelligent retry settings, test files that need to be rerun are detected, effectively ensuring the pass rate of test cases and reducing the difficulty in reproducing probabilistic failure cases.

[0163] As an optional implementation, the present application also provides a software testing system. Figure 3 is a schematic diagram of a software testing system according to an embodiment of the present application, such as Figure 3 As shown, the test system includes: an execution machine, a scheduler and a listener, wherein the execution machine is used to execute each test file in the test file set in the test environment in the current order under the control of the scheduler, wherein the test file set is used to test the software function of the target software; the listener is used to obtain the target execution information of the test file set and send it to the scheduler, wherein the target execution information is used to indicate the execution process and execution result of the test file set; the scheduler is used to generate environmental impact information according to the target execution information, wherein the environmental impact information is used to indicate the impact of executing each test file in the test file set on the test environment; according to the environmental impact information, the execution order of each test file in the current order is adjusted to obtain the target order, wherein the target order is used as the order of the next execution of the test file set. When the degree of impact of executing the first test file on the test environment indicated by the environmental impact information is greater than the degree of impact of executing the second test file on the test environment indicated by the environmental impact information, the execution order of the second test file in the target order precedes the execution order of the first test file.

[0164] Through the above test system, each test file in the test file set is executed in the target test environment according to the current order, and environmental impact information is generated according to the target execution information of the test file set to determine the impact of executing each test file in the test file set on the target test environment. Then, the execution order of each test file in the current order is adjusted according to the environmental impact information to obtain the order for executing the test file set for the next time. When the impact of executing the first test file on the target test environment indicated by the environmental impact information is greater than the impact of executing the second test file on the target test environment indicated by the environmental impact information, the execution order of the second test file in the target order precedes the execution order of the first test file, and the second test file with lower impact on the test environment is executed first, and then the first test file with higher impact on the environment is executed, so that the test environment can be used more and the waiting time for the execution of the test file to the test environment is reduced. Therefore, the technical problem of low software testing efficiency can be solved, and the technical effect of improving software testing efficiency can be achieved.

[0165] Optionally, in this embodiment, the scheduler is also used to: detect the test parameters of each test file to obtain multiple test parameters, wherein the test parameter is the ratio of the number of test cases that failed to execute in the test file to the total number of test cases included in the test file, and the target execution information includes the test parameter; generate environmental impact information of each test file according to each test parameter.

[0166] Optionally, in this embodiment, the scheduler is further used to: generate an impact identifier for the target test file corresponding to the target test parameter when the target test parameter among multiple test parameters is greater than or equal to a first threshold, wherein the impact identifier is used to indicate that executing the target test file will have an impact on the target test environment; determine the impact identifier and the continuous failure parameter as environmental impact information, wherein the continuous failure parameter is the maximum number of continuous failures in the target test file, and the number of continuous failures is the number of consecutive failures of each test case that failed to execute in the target test file in the historical time period before the current moment, and the target execution information also includes the continuous failure parameter.

[0167] Optionally, in this embodiment, the scheduler is further used to: when the environmental impact information includes an impact identifier, calculate the product of the current priority parameter and the adjustment parameter to obtain a growth parameter of the priority parameter, wherein the adjustment parameter is the quotient of the continuous failure parameter divided by the reduction coefficient, and the current order is generated based on the current priority parameters of each test file; calculate the sum of the growth parameter and the current priority parameter to obtain a target priority parameter; generate a target order based on each target priority parameter of each test file in the test file set, wherein the test file with a larger priority parameter has a later execution order in the target order.

[0168] Optionally, in this embodiment, the scheduler is also used to: generate an impact identifier for a reference test file corresponding to the reference test parameter when a reference test parameter among multiple test parameters is greater than a first threshold and a continuous failure parameter is greater than or equal to a second threshold, wherein the impact identifier is used to indicate that executing the reference test file will have an impact on the target test environment, the continuous failure parameter is the maximum number of consecutive failures in the reference test file, the number of consecutive failures is the number of consecutive failures of each test case that failed to execute in the reference test file in the historical time period before the current moment, and the target execution information also includes the continuous failure parameter; determining the environmental impact information includes the impact identifier and the reference test parameter.

[0169] Optionally, in this embodiment, the scheduler is further used to: when the environmental impact information includes an impact identifier, calculate the product of the reference test parameter and the amplification factor to obtain a growth parameter of the priority parameter; calculate the sum of the growth parameter and the current priority parameter to obtain a target priority parameter, wherein the current order is generated based on the current priority parameters of each test file; generate a target order based on each target priority parameter of each test file in the test file set, wherein the test file with a larger priority parameter has a later execution order in the target order.

[0170] Optionally, in this embodiment, the scheduler is also used to: calculate the sum of the growth parameter and the current priority parameter to obtain a preliminary priority parameter; when the preliminary priority parameter is greater than the priority parameter threshold of the test file set, determine the priority parameter threshold as the target priority parameter; when the preliminary priority parameter is less than or equal to the priority parameter threshold, determine the preliminary priority parameter as the target priority parameter.

[0171] Optionally, in this embodiment, the execution machine is further used to: match a corresponding target sub-environment for each test file from each sub-environment according to the environmental information of each sub-environment and the test requirements of each test file, wherein the environmental information is used to indicate the test conditions possessed by the sub-environment, the test requirements are used to indicate the test conditions required to execute the test file, and the target test environment includes each sub-environment; and execute each test file in the target sub-environment corresponding to each test file in the current order.

[0172] As an optional implementation, the test system also includes: a retryer; a listener, used to obtain reference execution information of each test file and send it to the retryer, wherein the reference execution information is used to indicate the execution results of each test case in the test file; a retryer, used to detect the reference execution information; in the case where it is detected that the reference execution information of the alternative test file is used to indicate that the third test case in the alternative test file has failed to execute, the retry requirement of the alternative test file is detected according to the test failure information of the third test case, wherein the test failure information is used to indicate that the third test case has failed to execute in the target number of executions before the current execution, and the retry requirement is the requirement for the alternative test file to be executed again; in the case where the retry requirement is that the alternative test file needs to be executed again, the execution machine is controlled to execute the alternative test file in the backup test environment.

[0173] Optionally, in this embodiment, the retryer is further configured to: when the number of test failures of the third test case is greater than the failure number threshold, calculate the confidence parameter of the third test case using the following formula:

[0174] ,

[0175] Among them, the confidence parameter X is used to indicate the randomness of the failure of the execution of the third test case, n is the target number, f is the number of test failures, the number of test failures is the number of times the third test case failed in the target number of executions before the current execution, the test failure information includes the number of test failures, and C (n, k) is the number of combinations; when the confidence parameter is greater than the confidence parameter threshold, the retry requirement is determined to be that the alternative test file needs to be executed again.

[0176] As an optional implementation, the present application also proposes an intelligent testing method based on environmental self-healing and dynamic priority. Figure 4 This is an architecture diagram of an intelligent testing method based on environmental self-healing and dynamic priority according to an embodiment of the present application, such as Figure 4 As shown, based on the TestNG framework (Test Next Generation, an open source automated testing framework), custom monitoring of test cases is managed. First, the priority of the test file class is dynamically adjusted. The historical execution records of the test suite are analyzed through the intelligent scheduling device, and the priority of the test class files with failed cases is lowered. Through the intelligent retry control device, the Bayesian algorithm is used to calculate the probability of failed cases, so as to obtain the test class files and reschedule them for operation. Through the test class file monitoring of the test environment, after the test case class file is executed, the environment self-recovery repair is performed when the release of the test environment fails.

[0177] Through the intelligent testing method based on environment self-healing and dynamic priority of the present application, for test case classes running in different types of test environments, an intelligent scheduling module is used for historical execution records to analyze the execution results of test class files. For use cases that have failed to run multiple times, resulting in residual test environments and abnormal release, the running priority of such test file classes is dynamically adjusted to the back, ensuring that more test class files are run in the test environment pool, thereby saving the running time of the test suite; retry control is performed on the test class files where the probabilistic failure use cases are located to improve the pass rate of the test suite; finally, self-healing is performed on the abnormal release of the test environment, reducing the failed use cases due to the test environment, improving the number of pass times of the test case class file use cases, and ensuring the calculation correctness of the intelligent module.

[0178] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method.

[0179] The embodiment of the present application also provides a software testing device, Figure 5 is a structural block diagram of a software testing device according to an embodiment of the present application, such as Figure 5 As shown, the device includes:

[0180] An execution module 502 is configured to execute each test file in a test file set in a target test environment according to a current order, wherein the test file set is used to test software functions of the target software;

[0181] A generating module 504 is configured to generate environmental impact information based on target execution information of the test file set, wherein the target execution information is used to indicate the execution process and execution results of the test file set, and the environmental impact information is used to indicate the impact of executing each test file in the test file set on the target test environment;

[0182] An adjustment module 506 is configured to adjust the execution order of each test file in the current sequence according to the environmental impact information to obtain a target sequence, wherein the target sequence serves as the sequence for executing the test file set next time. When the impact of executing the first test file on the target test environment indicated by the environmental impact information is greater than the impact of executing the second test file on the target test environment indicated by the environmental impact information, the execution order of the second test file in the target sequence precedes the execution order of the first test file.

[0183] Through the above device, each test file in the test file set is executed in the target test environment according to the current order, and environmental impact information is generated according to the target execution information of the test file set to determine the impact of executing each test file in the test file set on the target test environment. Then, the execution order of each test file in the current order is adjusted according to the environmental impact information to obtain the order for executing the test file set next time. When the impact of executing the first test file on the target test environment indicated by the environmental impact information is greater than the impact of executing the second test file on the target test environment indicated by the environmental impact information, the execution order of the second test file in the target order precedes the execution order of the first test file, and the second test file with lower impact on the test environment is executed first, and then the first test file with higher impact on the environment is executed, so that the test environment can be used more and the waiting time for the execution of the test file to the test environment is reduced. Therefore, the technical problem of low software testing efficiency can be solved, and the technical effect of improving software testing efficiency can be achieved.

[0184] Optionally, the generation module includes: a detection unit, used to detect the test parameters of each test file to obtain multiple test parameters, wherein the test parameter is the ratio of the number of test cases that failed to execute in the test file to the total number of test cases included in the test file, and the target execution information includes the test parameter; a first generation unit, used to generate environmental impact information of each test file based on each test parameter.

[0185] Optionally, the first generation unit is also used to: when a target test parameter among multiple test parameters is greater than or equal to a first threshold, generate an impact identifier for the target test file corresponding to the target test parameter, wherein the impact identifier is used to indicate that executing the target test file will have an impact on the target test environment; determine the impact identifier and the continuous failure parameter as environmental impact information, wherein the continuous failure parameter is the maximum number of continuous failures in the target test file, and the number of continuous failures is the number of consecutive failures of each test case that failed to execute in the target test file in the historical time period before the current moment, and the target execution information also includes the continuous failure parameter.

[0186] Optionally, the adjustment module includes: a first calculation unit, used to calculate the product of the current priority parameter and the adjustment parameter when the environmental impact information includes an impact identifier, to obtain a growth parameter of the priority parameter, wherein the adjustment parameter is the quotient of the continuous failure parameter divided by the reduction coefficient, and the current order is generated based on the current priority parameters of each test file; a second calculation unit, used to calculate the sum of the growth parameter and the current priority parameter, to obtain a target priority parameter; a second generation unit, used to generate a target order based on each target priority parameter of each test file in the test file set, wherein the test file with a larger priority parameter has a later execution order in the target order.

[0187] Optionally, the first generation unit is also used to: when a reference test parameter among multiple test parameters is greater than a first threshold and a continuous failure parameter is greater than or equal to a second threshold, generate an impact identifier of a reference test file corresponding to the reference test parameter, wherein the impact identifier is used to indicate that executing the reference test file will have an impact on the target test environment, the continuous failure parameter is the maximum number of consecutive failures in the reference test file, the number of consecutive failures is the number of consecutive failures of each test case that failed to execute in the reference test file in the historical time period before the current moment, and the target execution information also includes the continuous failure parameter; determining the environmental impact information includes the impact identifier and the reference test parameter.

[0188] Optionally, the adjustment module also includes: a third calculation unit, used to calculate the product of the reference test parameter and the amplification factor when the environmental impact information includes an impact identifier, to obtain a growth parameter of the priority parameter; a fourth calculation unit, used to calculate the sum of the growth parameter and the current priority parameter, to obtain a target priority parameter, wherein the current order is generated based on the current priority parameters of each test file; a third generation unit, used to generate a target order based on each target priority parameter of each test file in the test file set, wherein the test file with a larger priority parameter has a later execution order in the target order.

[0189] Optionally, the second calculation unit or the fourth calculation unit is further used to: calculate the sum of the growth parameter and the current priority parameter to obtain a preliminary priority parameter; when the preliminary priority parameter is greater than the priority parameter threshold of the test file set, determine the priority parameter threshold as the target priority parameter; when the preliminary priority parameter is less than or equal to the priority parameter threshold, determine the preliminary priority parameter as the target priority parameter.

[0190] Optionally, the execution module includes: a matching unit, used to match a corresponding target sub-environment for each test file from each sub-environment based on the environmental information of each sub-environment and the test requirements of each test file, wherein the environmental information is used to indicate the test conditions possessed by the sub-environment, the test requirements are used to indicate the test conditions required to execute the test file, and the target test environment includes each sub-environment; an execution unit, used to execute each test file in the target sub-environment corresponding to each test file in the current order.

[0191] Optionally, the software testing device further includes: a first detection module for detecting reference execution information of each test file, wherein the reference execution information is used to indicate the execution results of each test case in the test file; a second detection module for detecting a retry requirement of the alternative test file according to the test failure information of the third test case when it is detected that the reference execution information of the alternative test file is used to indicate that the execution of the third test case in the alternative test file has failed, wherein the test failure information is used to indicate that the third test case has failed in the target number of executions before the current execution, and the retry requirement is a requirement for the alternative test file to be executed again; a retry module for executing the alternative test file in the alternative test environment when the retry requirement is that the alternative test file needs to be executed again.

[0192] Optionally, the second detection module includes: a fifth calculation unit, configured to calculate the confidence parameter of the third test case by using the following formula when the number of test failures of the third test case is greater than the failure number threshold:

[0193] ,

[0194] Wherein, the confidence parameter X is used to indicate the randomness of the failure of the third test case execution, n is the target number, f is the number of test failures, the number of test failures is the number of times the third test case failed in the target number of executions before the current execution, the test failure information includes the number of test failures, and C(n,k) is the number of combinations;

[0195] The determining unit is used to determine that the retry requirement is that the alternative test file needs to be executed again when the confidence parameter is greater than the confidence parameter threshold.

[0196] For the description of the features in the embodiment corresponding to the software testing device, please refer to the relevant description of the embodiment corresponding to the software testing method, which will not be repeated here.

[0197] An embodiment of the present application further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any of the above software testing method embodiments.

[0198] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored. The computer program is configured to execute the steps of any of the above-mentioned software testing method embodiments when running.

[0199] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0200] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps in any of the above software testing method embodiments are implemented.

[0201] An embodiment of the present application further provides another computer program product, comprising a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any of the above-mentioned software testing method embodiments are implemented.

[0202] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0203] The above is a detailed introduction to the software testing method, system and device provided by the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core ideas of the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.< / string> < / xmlsuite>

Claims

1. A software testing method, characterized in that: include: Executing each test file in the test file set in the target test environment in the current order, wherein the test file set is used to test the software function of the target software; generating environmental impact information based on target execution information of the test file set, wherein the target execution information is used to indicate an execution process and an execution result of the test file set, and the environmental impact information is used to indicate an impact of executing each test file in the test file set on the target test environment; adjusting the execution order of each of the test files in the current order according to the environmental impact information to obtain a target order, wherein the target order serves as the order for the next execution of the set of test files, and if the impact of executing a first test file on the target test environment indicated by the environmental impact information is greater than the impact of executing a second test file on the target test environment indicated by the environmental impact information, the execution order of the second test file in the target order precedes the execution order of the first test file; The step of generating environmental impact information according to the target execution information of the test file set includes: detecting test parameters of each test file to obtain a plurality of test parameters, wherein the test parameter is a ratio of the number of test cases in the test file that failed to execute to the total number of test cases included in the test file, and the target execution information includes the test parameter; and generating environmental impact information for each test file according to each of the test parameters; The generating of the environmental impact information of each test file according to each test parameter includes: when a target test parameter among the multiple test parameters is greater than or equal to a first threshold value, generating an impact identifier of the target test file corresponding to the target test parameter, wherein the impact identifier is used to indicate that executing the target test file will have an impact on the target test environment; determining the impact identifier and a continuous failure parameter as the environmental impact information, wherein the continuous failure parameter is the maximum number of continuous failures in the target test file, and the number of continuous failures is the number of continuous failures of each test case that failed to execute in the target test file in a historical time period before the current moment, and the target execution information also includes the continuous failure parameter; Alternatively, when a reference test parameter among multiple test parameters is greater than a first threshold and a continuous failure parameter is greater than or equal to a second threshold, an impact identifier of the reference test file corresponding to the reference test parameter is generated, wherein the impact identifier is used to indicate that executing the reference test file will have an impact on the target test environment, the continuous failure parameter is the maximum number of continuous failures in the reference test file, the number of continuous failures is the number of consecutive failures of each test case that failed to execute in the reference test file within the historical time period before the current moment, and the target execution information also includes the continuous failure parameter; determining the environmental impact information includes the impact identifier and the reference test parameter.

2. The software testing method according to claim 1, characterized in that: The step of adjusting the execution order of each of the test files in the current order according to the environmental impact information to obtain a target order includes: In a case where the environmental impact information includes the impact identifier, calculating the product of a current priority parameter and an adjustment parameter to obtain a growth parameter of the priority parameter, wherein the impact identifier is generated when a target test parameter among the multiple test parameters is greater than or equal to a first threshold, the adjustment parameter is a quotient of the continuous failure parameter divided by a reduction coefficient, and the current order is generated according to the current priority parameters of each of the test files; Calculating the sum of the growth parameter and the current priority parameter to obtain a target priority parameter; The target order is generated according to the target priority parameters of the test files in the test file set, wherein the test file with a larger priority parameter has a later execution order in the target order.

3. The software testing method according to claim 1, characterized in that: The step of adjusting the execution order of each of the test files in the current order according to the environmental impact information to obtain a target order includes: In a case where the environmental impact information includes the impact identifier, calculating the product of the reference test parameter and the amplification factor to obtain a growth parameter of the priority parameter, wherein the impact identifier is generated when a reference test parameter among the plurality of test parameters is greater than a first threshold value and a continuous failure parameter is greater than or equal to a second threshold value; Calculating a sum of the growth parameter and a current priority parameter to obtain a target priority parameter, wherein the current order is generated according to the current priority parameter of each of the test files; The target order is generated according to the target priority parameters of the test files in the test file set, wherein the test file with a larger priority parameter has a later execution order in the target order.

4. The software testing method according to any one of claims 2 or 3, characterized in that: The calculating the sum of the growth parameter and the current priority parameter to obtain a target priority parameter includes: Calculating the sum of the growth parameter and the current priority parameter to obtain a preliminary priority parameter; In a case where the preliminary priority parameter is greater than a priority parameter threshold of the test file set, determining the priority parameter threshold as the target priority parameter; In a case where the preliminary priority parameter is less than or equal to the priority parameter threshold, the preliminary priority parameter is determined as the target priority parameter.

5. The software testing method according to claim 1, wherein: The executing each test file in the test file set in the target test environment according to the current order includes: Matching a corresponding target sub-environment for each test file from each sub-environment according to the environmental information of each sub-environment and the test requirements of each test file, wherein the environmental information is used to indicate the test conditions possessed by the sub-environment, the test requirements are used to indicate the test conditions required for executing the test file, and the target test environment includes each sub-environment; Execute each of the test files in the target sub-environment corresponding to each of the test files according to the current order.

6. The software testing method according to claim 1, characterized in that: In the process of executing each test file in the test file set in the target test environment according to the current order, the method further includes: Detecting reference execution information of each of the test files, wherein the reference execution information is used to indicate the execution result of each test case in the test file; In a case where it is detected that the reference execution information of the alternative test file is used to indicate that the third test case in the alternative test file has failed to execute, detecting a retry requirement of the alternative test file according to the test failure information of the third test case, wherein the test failure information is used to indicate that the third test case has failed to execute in a target number of executions before the current execution, and the retry requirement is a requirement for executing the alternative test file again; When the retry requirement is that the alternative test file needs to be executed again, the alternative test file is executed in the backup test environment.

7. The software testing method according to claim 6, characterized in that: The detecting the retry requirement of the alternative test file according to the test failure information of the third test case includes: When the number of test failures of the third test case is greater than the failure number threshold, the confidence parameter of the third test case is calculated using the following formula: ， The confidence parameter X is used to indicate the randomness of the failure of the execution of the third test case, n is the target number of times, f is the number of test failures, the number of test failures is the number of times the third test case failed to execute in the target number of executions before the current execution, the test failure information includes the number of test failures, C(n,k) is the number of combinations, k is the number of elements selected when calculating the number of combinations by selecting elements from n different elements; When the confidence parameter is greater than the confidence parameter threshold, it is determined that the retry requirement is that the alternative test file needs to be executed again.

8. A software testing system, characterized in that: The test system includes: an execution machine, a scheduler and a listener, wherein: The execution machine is configured to execute each test file in the test file set in a current order in a test environment under the control of the scheduler, wherein the test file set is used to test software functions of the target software; The listener is configured to obtain target execution information of the test file set and send the target execution information to the scheduler, wherein the target execution information is used to indicate the execution process and execution result of the test file set; The scheduler is configured to generate environmental impact information based on the target execution information, wherein the environmental impact information is used to indicate the impact of executing each test file in the test file set on the test environment; adjust the execution order of each test file in the current order based on the environmental impact information to obtain a target order, wherein the target order is used as the order for the next execution of the test file set, and if the degree of impact of executing a first test file on the test environment indicated by the environmental impact information is greater than the degree of impact of executing a second test file on the test environment indicated by the environmental impact information, the execution order of the second test file in the target order precedes the execution order of the first test file; The scheduler is further configured to: detect test parameters of each test file to obtain a plurality of test parameters, wherein the test parameter is a ratio of the number of test cases in the test file that failed to execute to the total number of test cases included in the test file, and the target execution information includes the test parameter; and generate environmental impact information for each test file according to each test parameter; The scheduler is also used to: In the case where a target test parameter among the multiple test parameters is greater than or equal to a first threshold, an impact identifier of a target test file corresponding to the target test parameter is generated, wherein the impact identifier is used to indicate that executing the target test file will have an impact on the test environment; the impact identifier and a continuous failure parameter are determined as the environmental impact information, wherein the continuous failure parameter is the maximum number of continuous failures in the target test file, and the number of continuous failures is the number of consecutive failures of each test case that failed to execute in the target test file in a historical time period before the current moment, and the target execution information also includes the continuous failure parameter; Alternatively, when a reference test parameter among multiple test parameters is greater than a first threshold and a continuous failure parameter is greater than or equal to a second threshold, an impact identifier of the reference test file corresponding to the reference test parameter is generated, wherein the impact identifier is used to indicate that executing the reference test file will have an impact on the test environment, the continuous failure parameter is the maximum number of continuous failures in the reference test file, the number of continuous failures is the number of consecutive failures of each test case that failed to execute in the reference test file within the historical time period before the current moment, and the target execution information also includes the continuous failure parameter; determining that the environmental impact information includes the impact identifier and the reference test parameter.

9. The software testing system according to claim 8, characterized in that: The test system further includes: a retryer; The listener is configured to obtain reference execution information of each of the test files and send the information to the retrieval device, wherein the reference execution information is used to indicate the execution result of each test case in the test file; The retryer is used to detect the reference execution information; when it is detected that the reference execution information of the alternative test file is used to indicate that the third test case in the alternative test file has failed to execute, the retry requirement of the alternative test file is detected according to the test failure information of the third test case, wherein the test failure information is used to indicate that the third test case has failed to execute in the target number of executions before the current execution, and the retry requirement is the requirement for the alternative test file to be executed again; when the retry requirement indicates that the alternative test file needs to be executed again, the execution machine is controlled to execute the alternative test file in the backup test environment.

10. An electronic device, characterized in that: include: memory for storing computer programs; A processor, configured to implement the steps of the software testing method according to any one of claims 1 to 7 when executing the computer program.

11. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the steps of the software testing method according to any one of claims 1 to 7 are implemented.

12. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the software testing method according to any one of claims 1 to 7 are implemented.

Images