Software testing method, system and device

Abstract

The invention discloses a software testing method, system and device, and relates to the technical field of software testing, and the method comprises the following steps: executing each test file in a test file set according to a current sequence in a target test environment; generating environment influence information according to the target execution information of the test file set; according to the environment influence information, the execution sequence of all the test files in the current sequence is adjusted, a target sequence is obtained, and the target sequence serves as the sequence of executing the test file set next time; under the condition that the influence, indicated by the environment influence information, of executing the first test file on the target test environment is larger than the influence, indicated by the environment influence information, of executing the second test file on the target test environment, the execution sequence of the second test file in the target sequence is earlier than the execution sequence of the first test file. By means of the software testing method and device, the technical problem that the software testing efficiency is low is solved, and the technical effect of improving the software testing efficiency is achieved.

Description

Technical Field

[0001] This application relates to the technical field of software testing, and particularly to a software testing method, system and device. Background Art

[0002] Software testing plays a very important role in ensuring software quality, reducing software costs, and improving user satisfaction. Currently, a testing scheme is adopted in which the entire set of test files corresponding to the software is executed in a test environment, and among them, the execution order of each test file in the test file set is random. This easily leads to a situation where some test files damage the test environment and then the subsequent test files lack a sufficient test environment for execution, thus resulting in a low software testing efficiency. Summary of the Invention

[0003] This application provides a software testing method, system and device to at least solve the problem of low software testing efficiency in related technologies.

[0004] This application provides a software testing method, including: executing each test file in the test file set in the target test environment according to the current order, where the test file set is used to test the software functions of the target software;

[0005] Generating environment impact information according to the target execution information of the test file set, where the target execution information is used to indicate the execution process and result of the test file set, and the environment impact information is used to indicate the impact of executing each test file in the test file set on the target test environment;

[0006] Adjusting the execution order of each test file in the current order according to the environment impact information to obtain the target order, where the target order is used as the order for executing the test file set next time. In the case where the impact of executing the first test file on the target test environment indicated by the environment impact information is greater than the impact of executing the second test file on the target test environment indicated by the environment impact information, the execution order of the second test file in the target order is prior to the execution order of the first test file.

[0007] This application provides a software testing system, which includes: an execution machine, a scheduler and a listener, where

[0008] The execution machine is used to execute each test file in the test file set in the test environment under the control of the scheduler, where the test file set is used to test the software functions of the target software;

[0009] The listener is used to obtain the target execution information of the test file set and send it to the scheduler, where the target execution information is used to indicate the execution process and result of the test file set;

[0010] A scheduler is configured to generate environmental impact information based on target execution information, where the environmental impact information is used to indicate the impact of each test file in the test file set on the test environment; adjust the execution order of each test file in the current order according to the environmental impact information to obtain a target order, where the target order is used as the order for the next execution of the test file set. When the impact degree of executing the first test file indicated by the environmental impact information on the test environment is greater than the impact degree of executing the second test file indicated by the environmental impact information on the test environment, the execution order of the second test file in the target order is prior to that of the first test file.

[0011] This application also provides an electronic device, including: a memory for storing a computer program; a processor for implementing the steps of any of the above software testing methods when executing the computer program.

[0012] This application also provides a computer-readable storage medium storing a computer program, where the computer program implements the steps of any of the above software testing methods when executed by a processor.

[0013] This application also provides a computer program product including a computer program, where the computer program implements the steps of any of the above software testing methods when executed by a processor.

[0014] Through this application, when executing each test file in the test file set in the target test environment according to the current order, environmental impact information is generated based on the target execution information of the test file set to determine the impact of each test file in the test file set on the target test environment, and then the execution order of each test file in the current order is adjusted according to the environmental impact information to obtain the order for the next execution of the test file set. When the impact of executing the first test file indicated by the environmental impact information on the target test environment is greater than the impact of executing the second test file indicated by the environmental impact information on the target test environment, the execution order of the second test file in the target order is prior to that of the first test file. The second test file with a lower impact on the test environment is executed first, and then the first test file with a higher impact on the environment is executed, so that the test environment can be used more, reducing the waiting for the test environment due to the execution of test files. Therefore, the technical problem of low software testing efficiency can be solved, and the technical effect of improving software testing efficiency can be achieved. Description of the Drawings

[0015] To more clearly illustrate the embodiments of the present application, the following will briefly introduce the accompanying drawings required for the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can be obtained based on these drawings.

[0016] Figure 1 is the hardware structure block diagram of the software testing method according to the embodiment of the present application;

[0017] Figure 2 is the flowchart of the software testing method according to the embodiment of the present application;

[0018] Figure 3 is the schematic diagram of a software testing system according to the embodiment of the present application;

[0019] Figure 4 is the architecture diagram of the intelligent testing method based on environment self-healing and dynamic priority according to the embodiment of the present application;

[0020] Figure 5 is the structure block diagram of a software testing device according to the embodiment of the present application. Specific Embodiments

[0021] The following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, rather than all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the protection scope of the present application.

[0022] It should be noted that in the description of the present application, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or further includes elements inherent to such process, method, article or device. The terms "first", "second", etc. in the present application are used to distinguish similar objects, rather than to describe a specific order or sequence.

[0023] To enable those skilled in the art of the present technology to better understand the solution of the present application, the following will further elaborate on the present application in combination with the accompanying drawings and specific embodiments.

[0024] Combined with the specific application environment architecture or specific hardware architecture on which the execution of the software testing method depends, the specific application environment architecture or specific hardware architecture is described herein.

[0025] The method embodiments provided in the embodiments of the present application may be executed in a server device or a similar computing device. Taking running on a server device as an example, Figure 1 is a hardware structural block diagram of the software testing method according to the embodiments of the present application. As Figure 1 shown, the server device may include one or more ( Figure 1 only one is shown in the figure) processors 102 (the processors 102 may include, but are not limited to, processing devices such as a microprocessor MCU or a field programmable gate array FPGA) and a memory 104 for storing data. Among them, the above-mentioned server device may further include a transmission device 106 for communication functions and an input / output device 108. Those of ordinary skill in the art can understand that Figure 1 the structure shown is only schematic and does not limit the structure of the above-mentioned server device. For example, the server device may further include more or fewer components than Figure 1 shown in the figure, or have a different configuration from Figure 1 shown in the figure.

[0026] The memory 104 can be used to store computer programs. For example, software programs and modules of application software, such as the computer program corresponding to the software testing method in the embodiments of the present application. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, the above-mentioned method is implemented. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some instances, the memory 104 may further include a memory remotely disposed relative to the processor 102, and these remote memories may be connected to the server device through a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an enterprise internal network, a local area network, a mobile communication network, and combinations thereof.

[0027] The transmission device 106 is used to receive or send data via a network. Specific examples of the above-mentioned network may include a wireless network provided by a communication provider of the server device. In one instance, the transmission device 106 includes a network adapter (abbreviated as NIC), which can be connected to other network devices through a base station and thus can communicate with the Internet. In one instance, the transmission device 106 may be a radio frequency (abbreviated as RF) module, which is used to communicate with the Internet wirelessly.

[0028] The embodiments of the present application provide a software testing method. In combination with the execution process of the software testing method, the method is described in detail.

[0029] In this embodiment, a software testing method is provided. Figure 2 It is a flowchart of the software testing method according to the embodiment of the present application. As Figure 2 shown, the method includes the following steps:

[0030] Step S202, execute each test file in the test file set in the target test environment in the current order, where the test file set is used to test the software functions of the target software;

[0031] Step S204, generate environment impact information according to the target execution information of the test file set, where the target execution information is used to indicate the execution process and execution result of the test file set, and the environment impact information is used to indicate the impact of executing each test file in the test file set on the target test environment;

[0032] Step S206, adjust the execution order of each test file in the current order according to the environment impact information to obtain the target order, where the target order is used as the order for the next execution of the test file set. In the case where the impact of executing the first test file indicated by the environment impact information on the target test environment is greater than the impact of executing the second test file indicated by the environment impact information on the target test environment, the execution order of the second test file in the target order is prior to the execution order of the first test file.

[0033] Through the above steps, each test file in the test file set is executed in the target test environment in the current order, environment impact information is generated according to the target execution information of the test file set to determine the impact of executing each test file in the test file set on the target test environment, and then the execution order of each test file in the current order is adjusted according to the environment impact information to obtain the order for the next execution of the test file set. In the case where the impact of executing the first test file indicated by the environment impact information on the target test environment is greater than the impact of executing the second test file indicated by the environment impact information on the target test environment, the execution order of the second test file in the target order is prior to the execution order of the first test file. The second test file with a lower impact on the test environment is executed first, and then the first test file with a higher impact on the environment is executed, so that the test environment can be used more, reducing the waiting for the test environment caused by the execution of the test file. Therefore, the technical problem of low software testing efficiency can be solved, and the technical effect of improving software testing efficiency can be achieved.

[0034] In the embodiment provided in step S202, the test environment may, but is not limited to, refer to a collection of hardware, software, network, etc. resources built for software testing, and may, but is not limited to, include: a hardware environment, a software environment, and a network environment. Among them, the hardware environment may, but is not limited to, include computer hardware (including servers, personal computers, etc.) and peripheral hardware (including printers, scanners, cameras, etc.), the software environment may, but is not limited to, include an operating system, a database system, middleware, etc., and the network environment may, but is not limited to, include a network topology structure (i.e., the connection method of computers, devices, etc. in the network) and network parameters (including bandwidth, latency, packet loss rate, etc.).

[0035] Optionally, in this embodiment, the target test environment may, but is not limited to, be a test environment that meets the requirements of the test file set for the test environment.

[0036] Optionally, in this embodiment, the test file set may, but is not limited to, be designed for the target software, and the test file set may, but is not limited to, be used to test the software functions of the target software.

[0037] Optionally, in this embodiment, the test file set may, but is not limited to, need to be executed multiple times in a loop in the target test environment to fully test the software functions of the target software.

[0038] Optionally, in this embodiment, during the first execution of the test file set, the current order may, but is not limited to, be a random order; during the non-first execution of the test file set, the current order may, but is not limited to, be an order determined according to the historical execution situation of the test file set.

[0039] Optionally, in this embodiment, the test file set may, but is not limited to, include multiple test files, and these test files are independent of each other, and the execution order of these test files does not affect the execution results of the test files.

[0040] In the embodiment provided in step S204, the target execution information may, but is not limited to, be used to indicate the execution process and execution results of the test file set, and the target execution information may, but is not limited to, include the execution start time, execution end time, execution results of each test file in the test file set, and the resource usage during the execution process (which may, but is not limited to, include CPU (Central Processing Unit, central processor) usage rate, memory usage, disk I / O (Input / Output, input / output), and network traffic), etc.

[0041] Optionally, in this embodiment, the environmental impact information can be used, but is not limited to, indicating the impact of each test file in the test file set on the target test environment. The impact of executing each test file on the target test environment can include, but is not limited to, causing some test environments in the target test environment to be abnormally occupied and unable to release the occupied environmental resources normally after the test file is executed, resulting in a reduction in the available test environments in the later stage.

[0042] Optionally, in this embodiment, the environmental impact information can include, but is not limited to, an impact identifier and a non-impact identifier. Among them, the impact identifier is used to indicate that executing the corresponding test file will have an impact on the target test environment, and the non-impact identifier is used to indicate that executing the corresponding test file will not have an impact on the target test environment. The environmental impact information can also include, but is not limited to, an environmental impact level, which is the level of the impact of executing the corresponding test file on the target test environment. The higher the environmental impact level, the greater the impact of executing the corresponding test file on the target test environment.

[0043] Optionally, in this embodiment, generating environmental impact information according to the target execution information of the test file set can include, but is not limited to, analyzing the resource usage during the execution of the test file set and evaluating the impact of the test file on the target test environment. Specifically, during the execution of each test file, data such as CPU usage rate, memory usage, disk I / O, and network traffic are collected in real time; the collected data is analyzed to calculate statistical indicators such as average values and peak values; the impact of the test file on the environment is evaluated based on the statistical indicators. For example, if the CPU usage rate of a certain test file continuously exceeds 80% during execution, it can be considered that the test file has a greater impact on the CPU resources of the environment. For example, for test file A and test file B, during the execution of test file A, the CPU usage rate reaches 90%, the memory usage is 4GB (Gigabyte), the disk I / O is frequent, and the network traffic is 10MB / s (Megabytes per second). During the execution of test file B, the CPU usage rate is 30%, the memory usage is 1GB, the disk I / O is less, and the network traffic is 1MB / s. It can be seen that test file A has a greater impact on the CPU and memory resources of the target test environment, which may cause the system to run slowly, and test file B has a smaller impact on the environment and the system runs more smoothly. Finally, it is obtained that the environmental impact level of test file A is higher than that of test file B.

[0044] In the embodiment provided in step S206, it is possible but not limited to deploy the execution order of test files that have a greater impact on the target test environment after execution according to the environmental impact information after the execution order of test files that have a smaller impact on the target test environment after execution, so as to ensure that the abnormal occupation of the target test environment appears in a later time period during the test process and improve the utilization rate of the target test environment.

[0045] Optionally, in this embodiment, adjusting the execution order of each test file in the current order according to the environmental impact information may but is not limited to including selecting test files that will have an impact on the target test environment after execution from the test set, and adjusting the execution order of the selected test files to the last position in the current order to obtain the target order; or, selecting test files that will have an impact on the target test environment after execution from the test set, sorting the selected test files according to the environmental impact level, and arranging the test files with higher environmental impact levels in a more backward position to obtain a test file adjustment sequence, and adjusting the execution order of the test file adjustment sequence to the last position in the current order to obtain the target order. For example, for a test file set including 4 test files, the current order of these 4 test files is Test File A - Test File B - Test File C - Test File D. When the current environmental impact information indicates that executing Test File B and Test File C will affect the target test environment, the current order is adjusted to Test File A - Test File D - Test File B - Test File C; when the current environmental impact information indicates that executing Test File B and Test File C will affect the target test environment and the environmental impact level of Test File B is higher than that of Test File C, the obtained test file adjustment sequence is Test File C - Test File B, and finally the current order is adjusted to Test File A - Test File D - Test File C - Test File B.

[0046] Optionally, in this embodiment, it is also possible but not limited to implement listening to IMethodInterceptor (an interface provided by TestNG (Test Next Generation, an open-source automated testing framework)) during the execution process of the test file set, perform exception handling on the test file to release the environment, and perform intelligent recovery according to the state of the environment before the test file is executed. By performing intelligent self-healing during the abnormal release process of the test file, reduce the failure of test cases caused by environmental interference and improve the passing rate of test case execution.

[0047] As an alternative implementation, generate environmental impact information based on the target execution information of the test file set, including: detecting the test parameters of each test file to obtain a plurality of test parameters, where the test parameter is the ratio of the number of successfully executed test cases in the test file to the total number of test cases included in the test file, and the target execution information includes the test parameter; generating environmental impact information for each test file according to each test parameter.

[0048] Optionally, in this embodiment, each test file may but is not limited to include one or more test cases. When a test file includes multiple test cases, there is an association relationship between the test cases. Therefore, the execution order of the multiple test cases in this test file is fixed.

[0049] Optionally, in this embodiment, the test parameter may but is not limited to be the ratio of the number of successfully executed test cases in the test file to the total number of test cases included in the test file. For example, when test file A includes b test cases and a test cases are successfully executed during a certain execution of test file A, the test parameter of test file A is a / b.

[0050] Optionally, in this embodiment, the test cases that fail to execute are likely to abnormally occupy the test resources of the target test environment. The environmental impact information for each test file may but is not limited to be generated according to each test parameter. The greater the test parameter of a test file, the greater the risk of affecting the target test environment.

[0051] Through the above content, judge the impact of each test file on the target test environment based on the execution situation of the test cases in each test file, and simply and feasibly determine the impact risk of the target test environment, providing strong effect support for reasonably adjusting the execution order of the test files.

[0052] As an alternative implementation, generating environmental impact information for each test file according to each test parameter includes: when the target test parameter among the multiple test parameters is greater than or equal to the first threshold, generating an impact identifier for the target test file corresponding to the target test parameter, where the impact identifier is used to indicate that executing the target test file will have an impact on the target test environment; determining the impact identifier and the consecutive failure parameter as the environmental impact information, where the consecutive failure parameter is the maximum number of consecutive failures in the target test file, and the number of consecutive failures is the number of consecutive failures of each test case that fails to execute in the target test file during the historical time period before the current moment. The target execution information further includes the consecutive failure parameter.

[0053] Optionally, in this embodiment, it is possible but not limited to determining that executing the target test file corresponding to the target test parameter will have an impact on the target test environment when the target test parameter s / n (where n is the total number of test cases included in the test file and s is the number of successfully executed test cases) is greater than or equal to the first threshold m, generating an impact identifier and determining the impact identifier and the consecutive failure parameter as the environmental impact information.

[0054] As an optional implementation manner, adjusting the execution order of each test file in the current order according to the environmental impact information to obtain the target order includes: when the environmental impact information includes an impact identifier, calculating the product of the current priority parameter and the adjustment parameter to obtain the growth parameter of the priority parameter, where the adjustment parameter is the quotient of the consecutive failure parameter divided by the reduction coefficient, and the current order is generated according to the current priority parameters of each test file; calculating the sum of the growth parameter and the current priority parameter to obtain the target priority parameter; generating the target order according to the respective target priority parameters of each test file in the test file set, where the test file with a larger priority parameter is arranged later in the target order.

[0055] Optionally, in this embodiment, it is possible but not limited to the target priority parameter P_new = P_old + (P_old × (x / 3)), where P_old is the current priority parameter, x / 3 is the adjustment parameter, x is the consecutive failure parameter, and 3 is the reduction coefficient.

[0056] Optionally, in this embodiment, each test file may have an initial default priority parameter, and it is possible but not limited to setting the default priority parameter to 5.

[0057] Optionally, in this embodiment, the adjustment range of the priority parameter of each test file may be limited, for example, the adjusted priority parameter may be less than or equal to the priority parameter threshold, and it is possible but not limited to setting the priority parameter threshold to 8.

[0058] Through the above content, the execution order of each test file in the test file set is adjusted, which can achieve executing the test file that is likely to affect the target test environment at a later position, reducing the abnormal occupation time of the target test environment and improving the test efficiency of the software.

[0059] As an alternative implementation, it can be but is not limited to the following input parameter definitions: P: the priority parameter of the test file (default value 5), s is the number of successfully executed test cases in the test file, n is the total number of test cases in the test file (s <= n), x is the number of consecutive failures (i.e., the consecutive failure parameter), and m is the failure rate threshold (default is 0.4). If s / n >= m, trigger the adjustment of the test file priority parameter. Specifically, the adjustment method is: the adjusted priority parameter is always greater than the default value, and the cumulative priority increment of the same test file does not exceed β (the upper limit can be but is not limited to set to 3) (i.e., the priority parameter threshold). The adjusted target priority parameter P_new = P_old + (P_old × (x / 3)), where P_old is the current priority parameter, x / 3 is the adjustment parameter, x is the consecutive failure parameter, and 3 is the reduction factor.

[0060] As an alternative implementation, it can be but is not limited to setting the default priority parameter of the test file (i.e., BASE_PRIORITY) to 5, setting the threshold y of the consecutive failure parameter (i.e., THRESHOLD) to 3, setting the highest priority parameter of the test file (i.e., MAX_PRIORITY) to 10, and setting the amplification parameter α (i.e., COEFFICIENT) to 2. When the consecutive failure parameter of the test file is greater than the threshold of the consecutive failure parameter, trigger the adjustment of the priority parameter. The adjusted target priority parameter P_new = P_old + α(x - y), where P_old is the current priority parameter. It should be noted that it is also necessary to ensure that the adjusted target priority parameter does not exceed the highest priority parameter of the test file. The following is the pseudo-code implementation of the foregoing method:

[0061] public class PriorityAdjuster implements IAnnotationTransformer {

[0062] private int calculateNewPriority(int x) {

[0063] final int BASE_PRIORITY = 5;

[0064] final int THRESHOLD = 3;

[0065] final int MAX_PRIORITY = 10;

[0066] final int COEFFICIENT = 2;

[0067] int delta = (x - PriorityConfig.FAILURE_THRESHOLD)

[0068] × PriorityConfig.PRIORITY_INCREMENT_COEFF;

[0069] return Math.min(

[0070] PriorityConfig.BASE_PRIORITY + delta,

[0071] PriorityConfig.MAX_PRIORITY );

[0073] }

[0074] }

[0075] As an alternative implementation, environmental impact information for each test file is generated based on each test parameter, including: when the reference test parameter among multiple test parameters is greater than the first threshold and the consecutive failure parameter is greater than or equal to the second threshold, an impact identifier for the reference test file corresponding to the reference test parameter is generated, where the impact identifier is used to indicate that executing the reference test file will have an impact on the target test environment, the consecutive failure parameter is the maximum number of consecutive failures in the reference test file, the number of consecutive failures is the number of consecutive failures of each test case with an execution failure in the reference test file within the historical time period before the current moment, and the target execution information further includes the consecutive failure parameter; it is determined that the environmental impact information includes the impact identifier and the reference test parameter.

[0076] Optionally, in this embodiment, it is possible but not limited to determining that executing the reference test file corresponding to the reference test parameter will have an impact on the target test environment when the reference test parameter is greater than the first threshold and the consecutive failure parameter is greater than or equal to the second threshold, generating the impact identifier, and determining the impact identifier and the reference test parameter as the environmental impact information.

[0077] Optionally, in this embodiment, it is possible but not limited to setting the first threshold to 0.4.

[0078] Optionally, in this embodiment, it is possible but not limited to setting the second threshold to 3.

[0079] As an alternative implementation, adjust the execution order of each test file in the current order according to the environmental impact information to obtain the target order, including: when the environmental impact information includes an impact identifier, calculate the product of the reference test parameter and the amplification factor to obtain the growth parameter of the priority parameter; calculate the sum of the growth parameter and the current priority parameter to obtain the target priority parameter, where the current order is generated according to the current priority parameters of each test file; generate the target order according to the target priority parameters of each test file in the test file set, where the test file with a larger priority parameter is placed later in the target order.

[0080] Optionally, in this embodiment, each test file may or may not have an initial default priority parameter, and the default priority parameter may or may not be set to 5.

[0081] Optionally, in this embodiment, the adjustment range of the priority parameter of each test file may or may not be limited, for example, the adjusted priority parameter may or may not be less than or equal to the priority parameter threshold, and the priority parameter threshold may or may not be set to 8.

[0082] Optionally, in this embodiment, the amplification factor may or may not be set to 2.5.

[0083] Optionally, in this embodiment, the generation of the foregoing target priority parameter may or may not be implemented through the following pseudocode:

[0084] / **

[0085] * Implement the IAlterSuiteListener interface for dynamic priority adjustment

[0086] * /

[0087] public class DynamicPriorityInjector implements IAlterSuiteListener {

[0088] private static final int X_THRESHOLD = 3; / / Threshold for consecutive failure times (i.e., the second threshold)

[0089] private static final double M_RATIO = 0.4; / / Failure rate threshold (i.e., the first threshold)

[0090] private static final int P_DEFAULT = 5; / / Default priority parameter

[0091] @Override

[0092] public void alter(List <xmlsuite>suites) {

[0093] Map<String, TestClassStats> stats = loadHistoryStats(); / / Load historical execution records

[0094] suites.forEach(suite -> {

[0095] suite.getTests().forEach(test -> {

[0096] test.getClasses().forEach(clazz -> {

[0097] XmlClass xmlClass = (XmlClass) clazz;

[0098] String className = xmlClass.getName();

[0099] if (stats.containsKey(className)) {

[0100] TestClassStats s = stats.get(className);

[0101] / / Condition judgment: consecutive failures x times and failure rate > m

[0102] if (s.consecutiveFails >= X_THRESHOLD

[0103] && (double)s.failedCases.size() / s.totalCases > M_RATIO) {

[0104] / / Calculate the new priority (i.e., calculate the target priority parameter)

[0105] int delta = (int)((s.failedCases.size() / (double)s.totalCases) * 2.5);

[0106] int newPriority = Math.min(P_DEFAULT + delta, P_DEFAULT + 3); / / Upper limit control

[0107] xmlClass.getMethods().forEach(m -> m.setPriority(newPriority));

[0108] }

[0109] }

[0110] });

[0111] });

[0112] });

[0113] }

[0114] / / Historical statistical data structure

[0115] private static class TestClassStats {

[0116] int consecutiveFails; / / Number of consecutive failures

[0117] Set <string>failedCases = new HashSet<>(); / / Set for failed test cases

[0118] int totalCases; / / Total number of test cases

[0119] }

[0120] }

[0121] Through the above content, the execution order of each test file in the test file set is adjusted, so that the test files that are likely to affect the target test environment are executed at a later position, reducing the abnormal occupancy time of the target test environment and improving the utilization efficiency of the target test environment.

[0122] As an optional implementation, calculate the sum of the growth parameter and the current priority parameter to obtain the target priority parameter, including: calculate the sum of the growth parameter and the current priority parameter to obtain the preliminary priority parameter; when the preliminary priority parameter is greater than the priority parameter threshold of the test file set, determine the priority parameter threshold as the target priority parameter; when the preliminary priority parameter is less than or equal to the priority parameter threshold, determine the preliminary priority parameter as the target priority parameter.

[0123] Optionally, in this embodiment, an upper limit for adjusting the priority parameter of the test file can be set, but is not limited to this, and the priority parameter cannot exceed the priority parameter threshold.

[0124] Optionally, in this embodiment, but not limited to this, when the preliminary priority parameter is greater than the priority parameter threshold of the test file set, an alarm message for the test file is generated, where the alarm message is used to indicate that the impact degree of the test file on the target test environment is abnormal, and the maintenance personnel will further analyze the abnormal cause of the test file after receiving the alarm message.

[0125] Through the above content, on the basis of adjusting the priority parameter to adjust the execution order of the test file, an upper limit for adjusting the priority parameter is also set, avoiding meaningless adjustment of the priority parameter when the priority parameter is too high.

[0126] As an optional implementation, execute each test file in the test file set in the current order in the target test environment, including: match the corresponding target sub-environment for each test file from each sub-environment according to the environment information of each sub-environment and the test requirements of each test file, where the environment information is used to indicate the test conditions possessed by the sub-environment, the test requirements are used to indicate the test conditions required for executing the test file, and the target test environment includes each sub-environment; execute each test file in the target sub-environment corresponding to each test file in the current order.

[0127] Optionally, in the embodiments of the present application, the target test environment may but is not limited to include multiple sub-environments. Different test conditions may but are not limited to be integrated in each sub-environment. The corresponding target sub-environment can be but is not limited to be matched for each test file from each sub-environment according to the test requirements of each test file, so as to implement the simultaneous execution of different test files in the test file set on multiple sub-environments and improve the test efficiency of the software.

[0128] As an alternative implementation manner, during the process of executing each test file in the test file set in the target test environment in the current order, the method further includes: detecting the reference execution information of each test file, where the reference execution information is used to indicate the execution results of each test case in the test file; when it is detected that the reference execution information of the alternative test file is used to indicate that the third test case in the alternative test file fails to execute, detecting the retry requirement of the alternative test file according to the test failure information of the third test case, where the test failure information is used to indicate the situation that the third test case fails to execute in the target number of executions before the current execution, and the retry requirement is the requirement for the alternative test file to be executed again; when the retry requirement is that the alternative test file needs to be executed again, executing the alternative test file in the alternative test environment.

[0129] Optionally, in this embodiment, it may but is not limited to detect the retry requirement of the alternative test file according to the test failure information of the third test case in the alternative test file when the third test case fails to execute continuously for multiple times. When the alternative test file needs to be executed again, execute the alternative test file in the alternative test environment.

[0130] Optionally, in this embodiment, the alternative test environment may but is not limited to be a test environment with the same test conditions as the target test environment.

[0131] Through the above content, it is determined whether a certain test case needs to be retried when the test case fails continuously. When it needs to be retried, the alternative test file is executed in the alternative test environment, avoiding the influence of the previously executed test file on the test environment on the execution result of the alternative test file, and obtaining a more accurate execution process and execution result of the alternative test file.

[0132] As an alternative implementation manner, detecting the retry requirement of the alternative test file according to the test failure information of the third test case includes: when the number of test failures of the third test case is greater than the failure number threshold, calculating the confidence parameter of the third test case through the following formula:

[0133] ,

[0134] Among them, the confidence parameter X is used to indicate the contingency of the failure of the third test case. n is the target number of times, f is the number of test failures. The number of test failures is the number of times the third test case fails during the execution of the target number of times before the current execution. The test failure information includes the number of test failures, and C(n,k) is the combination number. When the confidence parameter is greater than the confidence parameter threshold, it is determined that the retry requirement is that the alternative test file needs to be executed again.

[0135] Optionally, in this embodiment, it may be determined that the alternative test file needs to be re-executed in a clean and unaffected standby test environment when the number of test failures of the third test case is greater than the failure number threshold and the confidence parameter of the third test case is greater than the confidence parameter threshold.

[0136] Optionally, in this embodiment, the combination number C(n,k) is also called the binomial coefficient, which represents the number of ways to select k elements from n different elements. The combination here refers to the selection method without considering the order. For example, C(5,2)=10, which means there are 10 ways to select 2 items from 5 different items.

[0137] Optionally, in this embodiment, n can be set to 5, the failure number threshold m (i.e., DEFAULT_M) can be set to 2, and the confidence parameter threshold (i.e., CONFIDENCE_THRESHOLD) can be set to 0.6. The foregoing method can be implemented by the following pseudocode:

[0138] public class SmartRetryAnalyzer implements IRetryAnalyzer {

[0139] private static final int DEFAULT_M = 2;

[0140] private static final double CONFIDENCE_THRESHOLD = 0.6;

[0141] private ClassHistory classHistory;

[0142] @Override

[0143] public boolean retry(ITestResult result) {

[0144] / / 1. Get historical data

[0145] int n = classHistory.getRecentExecutionCount();

[0146] int f = classHistory.getRecentFailures();

[0147] / / 2. Basic condition judgment

[0148] if (n < 5 || f <= DEFAULT_M) return false;

[0149] / / 3. Calculate confidence

[0150] double confidence = ConfidenceCalculator.calculateConfidence(n, f);

[0151] / / 4. Dynamic decision-making

[0152] return confidence > CONFIDENCE_THRESHOLD;

[0153] }

[0154] / / Dynamic parameter configuration method

[0155] public void configure(

[0156] @Optional("5") int historySize,

[0157] @Optional("2") int failureThreshold,

[0158] @Optional("0.6") double confidenceThreshold) {

[0159] / / Parameter configuration logic

[0160] }

[0161] }

[0162] Through this application, a solution to the problem that it is not easy to reproduce the probabilistically failed test cases during the execution of a large number of test cases is proposed. By setting up intelligent retries, the test files that need to be re-run are detected, effectively ensuring the passing rate of test cases and reducing the problem of difficulty in reproducing probabilistically failed test cases.

[0163] As an alternative embodiment, the present application also provides a software testing system. Figure 3 It is a schematic diagram of a software testing system according to an embodiment of the present application. As Figure 3 shown, the testing system includes: an execution machine, a scheduler, and a listener. Among them, the execution machine is used to execute each test file in the test file set in the test environment in the current order under the control of the scheduler, where the test file set is used to test the software functions of the target software; the listener is used to obtain the target execution information of the test file set and send it to the scheduler, where the target execution information is used to indicate the execution process and execution result of the test file set; the scheduler is used to generate environment impact information according to the target execution information, where the environment impact information is used to indicate the impact of executing each test file in the test file set on the test environment; adjust the execution order of each test file in the current order according to the environment impact information to obtain a target order, where the target order is used as the order for the next execution of the test file set. In the case where the impact degree of executing the first test file indicated by the environment impact information on the test environment is greater than the impact degree of executing the second test file indicated by the environment impact information on the test environment, the execution order of the second test file in the target order is prior to the execution order of the first test file.

[0164] Through the above testing system, each test file in the test file set is executed in the current order in the target test environment, the environment impact information is generated according to the target execution information of the test file set to determine the impact of executing each test file in the test file set on the target test environment, and then the execution order of each test file in the current order is adjusted according to the environment impact information to obtain the order for the next execution of the test file set. In the case where the impact of executing the first test file indicated by the environment impact information on the target test environment is greater than the impact of executing the second test file indicated by the environment impact information on the target test environment, the execution order of the second test file in the target order is prior to the execution order of the first test file. The second test file with a lower impact on the test environment is executed first, and then the first test file with a higher impact on the environment is executed, so that the test environment can be used more, reducing the waiting of the test file execution for the test environment. Therefore, the technical problem of low software testing efficiency can be solved, and the technical effect of improving software testing efficiency can be achieved.

[0165] Optionally, in this embodiment, the scheduler is further used to: detect the test parameters of each test file to obtain a plurality of test parameters, where the test parameter is the ratio of the number of successfully executed test cases in the test file to the total number of test cases included in the test file, and the target execution information includes the test parameter; generate the environment impact information of each test file according to each test parameter.

[0166] Optionally, in this embodiment, the scheduler is further configured to: when the target test parameter among the multiple test parameters is greater than or equal to the first threshold, generate an impact identifier for the target test file corresponding to the target test parameter, where the impact identifier is used to indicate that executing the target test file will have an impact on the target test environment; determine the impact identifier and the consecutive failure parameter as the environment impact information, where the consecutive failure parameter is the maximum number of consecutive failures in the target test file, and the number of consecutive failures is the number of consecutive failures of each test case with an execution failure in the target test file within the historical time period before the current moment, and the target execution information further includes the consecutive failure parameter.

[0167] Optionally, in this embodiment, the scheduler is further configured to: when the environment impact information includes the impact identifier, calculate the product of the current priority parameter and the adjustment parameter to obtain the growth parameter of the priority parameter, where the adjustment parameter is the quotient of the consecutive failure parameter divided by the reduction factor, and the current order is generated according to the current priority parameters of each test file; calculate the sum of the growth parameter and the current priority parameter to obtain the target priority parameter; generate a target order according to the target priority parameters of each test file in the test file set, where the test file with a larger priority parameter is arranged later in the target order.

[0168] Optionally, in this embodiment, the scheduler is further configured to: when the reference test parameter among the multiple test parameters is greater than the first threshold and the consecutive failure parameter is greater than or equal to the second threshold, generate an impact identifier for the reference test file corresponding to the reference test parameter, where the impact identifier is used to indicate that executing the reference test file will have an impact on the target test environment, the consecutive failure parameter is the maximum number of consecutive failures in the reference test file, and the number of consecutive failures is the number of consecutive failures of each test case with an execution failure in the reference test file within the historical time period before the current moment, and the target execution information further includes the consecutive failure parameter; determine that the environment impact information includes the impact identifier and the reference test parameter.

[0169] Optionally, in this embodiment, the scheduler is further configured to: when the environment impact information includes the impact identifier, calculate the product of the reference test parameter and the amplification factor to obtain the growth parameter of the priority parameter; calculate the sum of the growth parameter and the current priority parameter to obtain the target priority parameter, where the current order is generated according to the current priority parameters of each test file; generate a target order according to the target priority parameters of each test file in the test file set, where the test file with a larger priority parameter is arranged later in the target order.

[0170] Optionally, in this embodiment, the scheduler is further configured to: calculate the sum of the growth parameter and the current priority parameter to obtain a preliminary priority parameter; determine the priority parameter threshold as the target priority parameter when the preliminary priority parameter is greater than the priority parameter threshold of the test file set; and determine the preliminary priority parameter as the target priority parameter when the preliminary priority parameter is less than or equal to the priority parameter threshold.

[0171] Optionally, in this embodiment, the executor is further configured to: match a corresponding target sub-environment for each test file from each sub-environment according to the environment information of each sub-environment and the test requirements of each test file, where the environment information is used to indicate the test conditions that the sub-environment has, the test requirements are used to indicate the test conditions required to execute the test file, and the target test environment includes each sub-environment; and execute each test file in the target sub-environment corresponding to each test file in the current order.

[0172] As an optional implementation manner, the test system further includes: a retrier; a listener, configured to obtain the reference execution information of each test file and send it to the retrier, where the reference execution information is used to indicate the execution results of each test case in the test file; the retrier, configured to detect the reference execution information; when it is detected that the reference execution information of the alternative test file is used to indicate that the third test case in the alternative test file fails to execute, detect the retry requirement of the alternative test file according to the test failure information of the third test case, where the test failure information is used to indicate the situation where the third test case fails to execute in the target number of executions before the current execution, and the retry requirement is the requirement for the alternative test file to be executed again; and when the retry requirement is that the alternative test file needs to be executed again, control the executor to execute the alternative test file in the standby test environment.

[0173] Optionally, in this embodiment, the retrier is further configured to: when the number of test failures of the third test case is greater than the failure number threshold, calculate the confidence parameter of the third test case through the following formula:

[0174] ,

[0175] where the confidence parameter X is used to indicate the contingency of the failure of the third test case to execute, n is the target number, f is the number of test failures, the number of test failures is the number of times the third test case fails to execute in the target number of executions before the current execution, the test failure information includes the number of test failures, C(n,k) is the combination number; and when the confidence parameter is greater than the confidence parameter threshold, determine that the retry requirement is that the alternative test file needs to be executed again.

[0176] As an optional implementation manner, the present application also proposes an intelligent testing method based on environment self-healing and dynamic priority. Figure 4 It is an architecture diagram of an intelligent testing method based on environment self-healing and dynamic priority according to an embodiment of the present application. As Figure 4 shown, based on the TestNG framework (Test Next Generation, an open-source automated testing framework), it manages the custom listening of test cases. First, it dynamically adjusts the priority of test file classes. By analyzing the historical execution records of test suites through an intelligent scheduling device, the priority of test class files with failed test cases is lowered; through an intelligent retry control device, the Bayesian algorithm is used to calculate the probability of failed test cases, thereby obtaining test class files and re-scheduling their execution; through listening to the test class files of the environment to be tested, when the environment to be tested fails to release after the execution of the test case class file is completed, the environment is self-healed and repaired.

[0177] Through an intelligent testing method based on environment self-healing and dynamic priority of the present application, for test case classes running in different types of environments to be tested, for historical execution records, an intelligent scheduling module is used to analyze the execution results of test class files. For test cases that have failed multiple times and cause the environment to be tested to remain and be abnormally released, the running priority of such test file classes is dynamically adjusted to be behind, ensuring that more test class files can run in the test environment pool in the environment to be tested, thereby saving the running time of test suites; for the test class files where the probability of failed test cases is located, retry control is performed to increase the passing rate of test suites; finally, the abnormal release of the environment to be tested is self-healed, reducing the number of failed test cases caused by the environment to be tested, improving the number of passing cases of test case class files, and ensuring the calculation correctness of the intelligent module.

[0178] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases, the former is a better implementation method.

[0179] The embodiment of the present application also provides a testing device for software, Figure 5 which is a structural block diagram of a testing device for software according to an embodiment of the present application. As Figure 5 shown, the device includes:

[0180] An execution module 502, configured to execute each test file in a test file set in a target test environment in the current order, where the test file set is used to test the software functions of the target software;

[0181] A generation module 504, configured to generate environment impact information according to the target execution information of the test file set, where the target execution information is used to indicate the execution process and execution results of the test file set, and the environment impact information is used to indicate the impact of executing each test file in the test file set on the target test environment;

[0182] An adjustment module 506 is configured to adjust the execution order of each test file in the current order according to the environment impact information to obtain a target order, where the target order is used as the order for executing the test file set next time. When the impact of executing the first test file indicated by the environment impact information on the target test environment is greater than the impact of executing the second test file indicated by the environment impact information on the target test environment, the execution order of the second test file in the target order is prior to that of the first test file.

[0183] By using the above device, each test file in the test file set is executed in the current order in the target test environment, environment impact information is generated according to the target execution information of the test file set to determine the impact of executing each test file in the test file set on the target test environment, and then the execution order of each test file in the current order is adjusted according to the environment impact information to obtain the order for executing the test file set next time. When the impact of executing the first test file indicated by the environment impact information on the target test environment is greater than the impact of executing the second test file indicated by the environment impact information on the target test environment, the execution order of the second test file in the target order is prior to that of the first test file. The second test file with a lower impact on the test environment is executed first, and then the first test file with a higher impact on the environment is executed, so that the test environment can be used more, and the waiting for the test environment caused by the execution of the test file is reduced. Therefore, the technical problem of low test efficiency of software can be solved, and the technical effect of improving the test efficiency of software can be achieved.

[0184] Optionally, the generation module includes: a detection unit configured to detect the test parameters of each test file to obtain a plurality of test parameters, where the test parameter is the ratio of the number of successfully executed test cases in the test file to the total number of test cases included in the test file, and the target execution information includes the test parameter; a first generation unit configured to generate the environment impact information of each test file according to each test parameter.

[0185] Optionally, the first generation unit is further configured to: when the target test parameter in the plurality of test parameters is greater than or equal to a first threshold, generate an impact identifier corresponding to the target test file, where the impact identifier is used to indicate that executing the target test file will have an impact on the target test environment; determine the impact identifier and the consecutive failure parameter as the environment impact information, where the consecutive failure parameter is the maximum number of consecutive failures in the target test file, and the number of consecutive failures is the number of consecutive failures of each failed test case in the target test file in the historical time period before the current moment, and the target execution information further includes the consecutive failure parameter.

[0186] Optionally, the adjustment module includes: a first calculation unit, configured to calculate the product of the current priority parameter and the adjustment parameter to obtain a growth parameter of the priority parameter when the environmental impact information includes an impact identifier, where the adjustment parameter is the quotient of the consecutive failure parameter divided by the reduction factor, and the current order is generated according to the current priority parameters of the respective test files; a second calculation unit, configured to calculate the sum of the growth parameter and the current priority parameter to obtain a target priority parameter; a second generation unit, configured to generate a target order according to the respective target priority parameters of the respective test files in the test file set, where the test file with a larger priority parameter has a later execution order in the target order.

[0187] Optionally, the first generation unit is further configured to: generate an impact identifier of the reference test file corresponding to the reference test parameter when the reference test parameter in the multiple test parameters is greater than the first threshold and the consecutive failure parameter is greater than or equal to the second threshold, where the impact identifier is used to indicate that executing the reference test file will have an impact on the target test environment, the consecutive failure parameter is the maximum consecutive failure times in the reference test file, and the consecutive failure times are the consecutive failure times of the respective test cases with execution failures in the reference test file within the historical time period before the current moment, and the target execution information further includes the consecutive failure parameter; determine that the environmental impact information includes the impact identifier and the reference test parameter.

[0188] Optionally, the adjustment module further includes: a third calculation unit, configured to calculate the product of the reference test parameter and the amplification factor to obtain a growth parameter of the priority parameter when the environmental impact information includes an impact identifier; a fourth calculation unit, configured to calculate the sum of the growth parameter and the current priority parameter to obtain a target priority parameter, where the current order is generated according to the current priority parameters of the respective test files; a third generation unit, configured to generate a target order according to the respective target priority parameters of the respective test files in the test file set, where the test file with a larger priority parameter has a later execution order in the target order.

[0189] Optionally, the second calculation unit or the fourth calculation unit is further configured to: calculate the sum of the growth parameter and the current priority parameter to obtain a preliminary priority parameter; determine the priority parameter threshold as the target priority parameter when the preliminary priority parameter is greater than the priority parameter threshold of the test file set; determine the preliminary priority parameter as the target priority parameter when the preliminary priority parameter is less than or equal to the priority parameter threshold.

[0190] Optionally, the execution module includes: a matching unit configured to match a corresponding target sub-environment for each test file from each sub-environment according to the environment information of each sub-environment and the test requirements of each test file, where the environment information is used to indicate the test conditions available in the sub-environment, the test requirements are used to indicate the test conditions required for executing the test file, and the target test environment includes each sub-environment; an execution unit configured to execute each test file in the target sub-environment corresponding to each test file in the current order.

[0191] Optionally, the test apparatus for software further includes: a first detection module configured to detect the reference execution information of each test file, where the reference execution information is used to indicate the execution results of each test case in the test file; a second detection module configured to, when detecting that the reference execution information of an alternative test file indicates that a third test case in the alternative test file fails to execute, detect the retry requirement of the alternative test file according to the test failure information of the third test case, where the test failure information is used to indicate the situation where the third test case fails to execute in the target number of executions before the current execution, and the retry requirement is the requirement for the alternative test file to be executed again; a retry module configured to execute the alternative test file in a standby test environment when the retry requirement is that the alternative test file needs to be executed again.

[0192] Optionally, the second detection module includes: a fifth calculation unit configured to, when the number of test failures of the third test case is greater than the failure number threshold, calculate the confidence parameter of the third test case through the following formula:

[0193] ,

[0194] where the confidence parameter X is used to indicate the contingency of the failure of the third test case to execute, n is the target number, f is the number of test failures, the number of test failures is the number of times the third test case fails to execute in the target number of executions before the current execution, the test failure information includes the number of test failures, and C(n,k) is the combination number;

[0195] a determination unit configured to determine that the retry requirement is that the alternative test file needs to be executed again when the confidence parameter is greater than the confidence parameter threshold.

[0196] For the description of the features in the embodiments corresponding to the test apparatus for software, reference can be made to the relevant descriptions in the embodiments corresponding to the test method for software, which will not be elaborated here one by one.

[0197] An embodiment of the present application further provides an electronic device, including a memory and a processor. A computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any of the above embodiments of the test method for software.

[0198] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored. Wherein, the computer program is configured to execute the steps in any of the above-described embodiments of the software testing method when running.

[0199] In an exemplary embodiment, the above computer-readable storage medium may include, but is not limited to: various media such as USB flash drives, read-only memories (ROM for short), random access memories (RAM for short), external hard drives, magnetic disks, or optical discs that can store computer programs.

[0200] An embodiment of the present application further provides a computer program product. The above computer program product includes a computer program, and when the computer program is executed by a processor, it implements the steps in any of the above-described embodiments of the software testing method.

[0201] An embodiment of the present application further provides another computer program product, including a non-volatile computer-readable storage medium. The non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements the steps in any of the above-described embodiments of the software testing method.

[0202] Those skilled in the art can further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Skilled professionals can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0203] The above has introduced in detail a software testing method, system, and device provided by the present application. Specific examples are used herein to elaborate on the principle and implementation manner of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application. It should be noted that for those of ordinary skill in the art in this technical field, without departing from the principle of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the protection scope of the claims of the present application.< / string> < / xmlsuite>

Claims

1. A software testing method, characterized in that, Including: Executing each test file in the test file set in the target test environment in the current order, where the test file set is used to test the software functions of the target software; Generating environment impact information according to the target execution information of the test file set, where the target execution information is used to indicate the execution process and execution results of the test file set, and the environment impact information is used to indicate the impact of executing each test file in the test file set on the target test environment; Adjusting the execution order of each test file in the current order according to the environment impact information to obtain a target order, where the target order is used as the order for executing the test file set next time. When the impact of executing the first test file indicated by the environment impact information on the target test environment is greater than the impact of executing the second test file indicated by the environment impact information on the target test environment, the execution order of the second test file is prior to the execution order of the first test file in the target order.

2. The software testing method according to claim 1, wherein The generating environment impact information according to the target execution information of the test file set includes: Detecting the test parameters of each test file to obtain a plurality of test parameters, where the test parameter is the ratio of the number of successfully executed test cases in the test file to the total number of test cases included in the test file, and the target execution information includes the test parameter; Generating the environment impact information of each test file according to each test parameter.

3. The testing method of the software according to claim 2, characterized in that The generating the environment impact information of each test file according to each test parameter includes: When the target test parameter in the plurality of test parameters is greater than or equal to a first threshold, generating an impact identifier corresponding to the target test file, where the impact identifier is used to indicate that executing the target test file will have an impact on the target test environment; Determining the impact identifier and the consecutive failure parameter as the environment impact information, where the consecutive failure parameter is the maximum number of consecutive failures in the target test file, and the number of consecutive failures is the number of consecutive failures of each failed test case in the target test file during the historical time period before the current moment, and the target execution information further includes the consecutive failure parameter.

4. The software testing method according to claim 3, wherein The adjusting the execution order of each test file in the current order according to the environment impact information to obtain a target order includes: When the environment impact information includes the impact identifier, calculating the product of the current priority parameter and the adjustment parameter to obtain a growth parameter of the priority parameter, where the adjustment parameter is the quotient of the consecutive failure parameter divided by a reduction factor, and the current order is generated according to the current priority parameter of each test file; Calculating the sum of the growth parameter and the current priority parameter to obtain a target priority parameter; Generating the target order according to each of the target priority parameters of each of the test files in the test file set, wherein the test file with a larger priority parameter is arranged later in the execution order in the target order.

5. The testing method of the software according to claim 2, characterized in that, Generating the environmental impact information of each of the test files according to each of the test parameters, including: When a reference test parameter in a plurality of the test parameters is greater than a first threshold and a consecutive failure parameter is greater than or equal to a second threshold, generating an impact identifier for a reference test file corresponding to the reference test parameter, where the impact identifier is used to indicate that executing the reference test file will have an impact on the target test environment, the consecutive failure parameter is the maximum number of consecutive failures in the reference test file, and the number of consecutive failures is the number of consecutive failures of each test case that fails to execute in the reference test file within a historical time period before the current moment, and the target execution information further includes the consecutive failure parameter; Determining that the environmental impact information includes the impact identifier and the reference test parameter.

6. The software testing method according to claim 5, wherein Adjusting the execution order of each of the test files in the current order according to the environmental impact information to obtain a target order, including: When the environmental impact information includes the impact identifier, calculating a product of the reference test parameter and a magnification factor to obtain a growth parameter of the priority parameter; Calculating a sum of the growth parameter and a current priority parameter to obtain a target priority parameter, where the current order is generated according to the current priority parameters of each of the test files; Generating the target order according to each of the target priority parameters of each of the test files in the test file set, wherein the test file with a larger priority parameter is arranged later in the execution order in the target order.

7. The test method of software according to any one of claims 4 or 6, characterized in that The calculating a sum of the growth parameter and the current priority parameter to obtain a target priority parameter includes: Calculating a sum of the growth parameter and the current priority parameter to obtain a preliminary priority parameter; When the preliminary priority parameter is greater than a priority parameter threshold of the test file set, determining the priority parameter threshold as the target priority parameter; When the preliminary priority parameter is less than or equal to the priority parameter threshold, determining the preliminary priority parameter as the target priority parameter.

8. The software testing method according to claim 1, characterized in that, Executing each of the test files in the test file set in the target test environment according to the current order, including: Matching a corresponding target sub-environment for each of the test files from each of the sub-environments according to the environmental information of each sub-environment and the test requirements of each of the test files, where the environmental information is used to indicate the test conditions that the sub-environment possesses, the test requirements are used to indicate the test conditions required for executing the test file, and the target test environment includes each of the sub-environments; Executing each of the test files in the corresponding target sub-environment of each of the test files according to the current order.

9. The software testing method according to claim 1, wherein During the process of executing each test file in the test file set in the target test environment in the current order, the method further includes: Detecting the reference execution information of each of the test files, where the reference execution information is used to indicate the execution results of each test case in the test file; When it is detected that the reference execution information of the alternative test file is used to indicate that the third test case in the alternative test file fails to execute, detecting the retry requirement of the alternative test file according to the test failure information of the third test case, where the test failure information is used to indicate the situation where the third test case fails to execute in the target number of executions before the current execution, and the retry requirement is the requirement for the alternative test file to be executed again; When the retry requirement is that the alternative test file needs to be executed again, execute the alternative test file in the standby test environment.

10. The testing method of the software according to claim 9, wherein The detecting the retry requirement of the alternative test file according to the test failure information of the third test case includes: When the number of test failures of the third test case is greater than the failure number threshold, calculate the confidence parameter of the third test case through the following formula: ， where the confidence parameter X is used to indicate the contingency of the failure of the third test case to execute, n is the target number, f is the number of test failures, the number of test failures is the number of times the third test case fails to execute in the target number of executions before the current execution, the test failure information includes the number of test failures, and C(n,k) is the combination number; When the confidence parameter is greater than the confidence parameter threshold, determine that the retry requirement is that the alternative test file needs to be executed again.

11. A testing system for software, characterized in that, The test system includes: an executor, a scheduler, and a listener, where The executor is used to execute each test file in the test file set in the test environment in the current order under the control of the scheduler, where the test file set is used to test the software functions of the target software; The listener is used to obtain the target execution information of the test file set and send it to the scheduler, where the target execution information is used to indicate the execution process and execution results of the test file set; The scheduler is used to generate environment impact information according to the target execution information, where the environment impact information is used to indicate the impact of executing each test file in the test file set on the test environment; adjusting the execution order of each test file in the current order according to the environment impact information to obtain a target order, where the target order is used as the order for the next execution of the test file set, and when the degree of impact of executing the first test file indicated by the environment impact information on the test environment is greater than the degree of impact of executing the second test file indicated by the environment impact information on the test environment, in the target order, the execution order of the second test file is prior to the execution order of the first test file.

12. The test system for software according to claim 11, wherein The test system further includes: a retryer; The listener is used to obtain the reference execution information of each of the test files and send it to the retrier, where the reference execution information is used to indicate the execution results of each test case in the test file; The retrier is used to detect the reference execution information; when it is detected that the reference execution information of the alternative test file is used to indicate that the third test case in the alternative test file fails to execute, detect the retry requirement of the alternative test file according to the test failure information of the third test case, where the test failure information is used to indicate the situation where the third test case fails to execute in the target number of executions before the current execution, and the retry requirement is the requirement for the alternative test file to be executed again; when the retry requirement is that the alternative test file needs to be executed again, control the execution machine to execute the alternative test file in the alternative test environment.

13. An electronic device, characterized in that, including: A memory for storing computer programs; A processor, when executing the computer program, is used to implement the steps of the software testing method according to any one of claims 1 to 10.

14. A computer-readable storage medium, characterized in that, A computer program is stored in the computer-readable storage medium, where when the computer program is executed by a processor, the steps of the software testing method according to any one of claims 1 to 10 are implemented.

15. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, the steps of the software testing method according to any one of claims 1 to 10 are implemented.

Images