Automatic test grading retry control method, system, equipment and medium

Abstract

The invention provides an automatic test grading retry control method, system and device and a medium, and belongs to the technical field of software test.The method comprises the steps that a configuration file is added in an automatic test framework, and a failure retry strategy is configured for a test script; before the automatic test is executed, configuring grades for the test script; in the execution process of the test script, if abnormal information is captured, matching the abnormal information with an abnormal type in a failure retry strategy through a regular expression; executing a failure retry strategy according to a matching result: if the matching succeeds, selecting the corresponding maximum retry times according to the grade of the test script and executing the retry, and if the test script does not have the configured grade, executing the retry according to the global default times; if matching fails, retry is prohibited; and adding the number of retry times in the execution of the test script into the test report. According to the method, the retry strategy can be adjusted according to different exception types and the priorities of the test scripts, resource waste is avoided, and meanwhile the real-time performance of the test strategy is ensured through dynamic updating.

Description

Technical Field

[0001] This application belongs to the technical field of software testing, and particularly relates to an automated test hierarchical retry control method, system, device, and medium. Background Art

[0002] With the increasingly widespread use of software applications, the complexity and functionality of software products are constantly increasing, and software automated testing is becoming increasingly important in the software development process. Automated testing can efficiently execute test cases, thereby quickly discovering software defects and improving software quality. However, during the automated testing process, the execution of test scripts may fail for various reasons, such as network problems, incorrect environment configurations, and code defects. To ensure the accuracy and integrity of testing, failed test scripts need to be retried in automated testing.

[0003] However, traditional automated test retries usually adopt a fixed number of retries and execute a unified strategy for all failed test scripts. This approach has the following problems: Firstly, it is a waste of resources. For some simple and easily solvable problems, multiple retries may not be required, while for some complex problems, the fixed number of retries may not be sufficient to solve them, resulting in a waste of test resources. Secondly, it lacks flexibility and cannot formulate differentiated retry strategies according to different exception types and the priorities of test scripts, making it difficult to adapt to changing test scenarios. Finally, it has poor dynamic adjustment ability. During the software development process, the code is constantly updated, and the test environment may also change. Traditional retry strategies are difficult to flexibly adjust according to these changes and lack dynamic configuration capabilities. Summary of the Invention

[0004] In a first aspect, an embodiment of this application provides an automated test hierarchical retry control method, including the following steps: S1. Add a configuration file to the automated test framework to configure a failure retry strategy for the test script: Configure the global default number of times; The maximum number of retries for each level of test script; The exception types allowed for retries; S2. Before executing the automated test, configure the level of the test script; S3. During the execution of the test script, if an exception message is captured, match the exception message with the exception types in the failure retry strategy through regular expressions; S4. Execute the failure retry strategy according to the matching result: If the match is successful, select the corresponding maximum number of retries according to the level of the test script and execute. If the test script is not configured with a level, retry according to the global default number of times; If the match fails, prohibit retries; S5. Add the number of retries during the execution of the test script to the test report.

[0005] Further, the specific steps of step S1 are as follows: S11. Build an automated test framework based on the Python + Pytest + selenium language; S12. Write a failure retry strategy file in yaml format and add it to the automated test framework.

[0006] Further, the specific steps of step S2 are as follows: S21. Separate the test cases and test data of the test script under the automated test framework and save the test data to an Excel data table; S22. Determine whether each test case of the test script needs to add a level configuration; If yes, go to step S23; If no, go to step S24; S23. Add a level configuration to the test case in the form of an annotation or label; S24. Add the test case to the automated test framework.

[0007] Further, the specific steps of step S3 are as follows: S31. Under the automated test framework, import the test data for the test case and execute it; S32. Determine whether exception information is captured during the execution of the test case; If no, wait for a set period of time and return to step S32; If yes, go to step S33; S33. Convert the exception information into a string form and use a regular expression to match it with the exception types allowed for retry in the failure retry strategy.

[0008] Further, the specific steps of step S4 are as follows: S41. Judge the matching result of the regular expression; If the match fails, mark the test item of the test case as a test failure and go to step S46; If the match is successful, go to step S42; S42. Determine whether the test case has a level configuration added; If yes, use the maximum number of retries for the corresponding level as the number of retries allowed; If no, use the global default number as the number of retries allowed; S43. Execute a retry for the corresponding test item of the test case and increment the number of retries by 1; S44. Determine whether the corresponding test item of the test case is successfully executed; If so, proceed to step S46; If not, proceed to step S45; S45. Determine whether the number of retries exceeds the maximum number of retries allowed; If so, mark the test item of the test case as a test failure and proceed to step S5; If not, return to step S43; S46. Determine whether the current test case has been executed completely; If so, proceed to step S5; If not, continue to execute the test case and return to step S3.

[0009] Furthermore, the following steps are also included: Under the automated test framework, create a policy object for the failure retry policy file through global variables and integrate a file system monitoring component; During the execution of the test script, use the file system monitoring component to monitor the modification events of the failure retry policy file in yaml format in real time; When the modification event is detected, automatically perform configuration parsing, use the PyYAML library to reload the updated failure retry policy file, and replace the old failure retry policy in memory.

[0010] Furthermore, the failure retry policy also includes the following content: Execute the calibrated number of retries for the first exception type; Increase the number of retries for the second exception type; During the execution of the test script, If the exception information matches the first exception type, use the calibrated number of retries to overwrite the number of retries allowed; If the exception information matches the second exception type, add the number of retries allowed to the increased number of retries.

[0011] In a second aspect, an automated test hierarchical retry control system provided by an embodiment of the present application includes: A retry policy configuration module, used to add a configuration file in the automated test framework and configure a failure retry policy for the test script: Configure the global default number; The maximum number of retries for each level of test script; The exception types allowed to be retried; A script level configuration module, used to configure the level of the test script before performing the automated test; A failure retry policy matching module, which is used to match the exception information with the exception type in the failure retry policy through regular expressions when an exception information is captured during the execution of the test script; A failure retry policy execution module, which is used to execute the failure retry policy according to the matching result: If the matching is successful, select the corresponding maximum number of retries according to the level of the test script and execute. If the test script does not configure a level, retry according to the global default number of times; If the matching fails, retry is prohibited; A test report generation module, which is used to add the number of retries during the execution of the test script to the test report.

[0012] Thirdly, an embodiment of the present application further provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the steps of the automated test hierarchical retry control method described in the first aspect are implemented.

[0013] Fourthly, an embodiment of the present application further provides a storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the automated test hierarchical retry control method described in the first aspect are implemented.

[0014] It can be seen from the above technical solutions that the present application has the following advantages: In the automated test hierarchical retry control method, system, device and medium provided by the present application, different levels of test script retry policies are flexibly defined through a configuration file, and the exception type is accurately matched by using regular expressions to execute different retry logics, and the policy can also be dynamically updated. It not only avoids resource waste, but also improves the accuracy and flexibility of testing, and is especially suitable for complex software development scenarios. Description of the Drawings

[0015] In order to more clearly illustrate the technical solutions of the present application, the drawings required to be used in the description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0016] Figure 1 It is a flowchart of the automated test hierarchical retry control method of the present invention.

[0017] Figure 2 It is a schematic diagram of the automated test hierarchical retry control system of the present invention. Detailed Embodiments

[0018] In the following detailed description of the specific steps of the automated test hierarchical retry control method, various embodiments of the present disclosure will be described more comprehensively. The present disclosure can have various embodiments, and adjustments and changes can be made therein. However, it should be understood that there is no intention to limit the various embodiments of the present disclosure to the specific embodiments disclosed herein, but the present disclosure should be understood to cover all adjustments, equivalents, and / or alternative options that fall within the spirit and scope of the various embodiments of the present disclosure.

[0019] Exemplarily speaking, in today's digital age, the popularity of software applications is increasing day by day, and the complexity and functionality of software products are constantly climbing. Software automated testing has also become increasingly crucial in the software development process. Automated testing can quickly execute test cases with its high efficiency, and can timely detect software vulnerabilities, thereby improving software quality. However, during the automated testing process, the situation where test scripts fail to execute occurs frequently, and the reasons are diverse, covering network fluctuations, environmental configuration deviations, and code defects, etc. To ensure the accuracy and comprehensiveness of testing, retrying the failed test scripts has become an essential part of automated testing.

[0020] However, the conventional automated test retry mechanism usually adopts a fixed number of retries and applies a unified strategy to all failed scripts. This mode has the following disadvantages: First, resource waste. For some simple and easily solved problems, there may be no need to repeat the attempt multiple times; while for complex problems, the fixed number of retries may be insufficient and unable to completely solve the problem, thus causing unreasonable consumption of test resources. Second, lack of flexibility. It is difficult to customize different retry schemes according to different types of exceptions and the priorities of test scripts, so it is impossible to cope with changing test scenarios. Third, poor dynamic adjustment ability. During the software development process, the code is continuously iterated and updated, and the test environment may also change accordingly. The traditional retry strategy is difficult to adapt to these changes, lacks the dynamic configuration ability of flexible adjustment, and is difficult to meet the rapidly changing needs in modern software development.

[0021] In view of the above problems, this embodiment provides an automated test hierarchical retry control method, which can flexibly adjust the retry strategy according to different types of exceptions and the priorities of test scripts, avoiding resource waste, and at the same time ensuring the real-time nature of the test strategy through dynamic update to adapt to the rapid changes in the software development process.

[0022] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0023] Please refer toFigure 1 The following is a flowchart of an automated test hierarchical retry control method in a specific embodiment. The method includes the following steps: S1. Add a configuration file in the automated test framework to configure the failure retry policy for the test script: Configure the global default number of times; The maximum number of retries for test scripts of each level; The types of exceptions allowed for retries; It should be noted that by adding a configuration file in the automated test framework to centrally manage the failure retry policy, the global default number of times, the maximum number of retries for test scripts of each level, and the types of exceptions allowed for retries can be flexibly configured, realizing fine management of test scripts, improving the flexibility and pertinence of test execution, reducing resource consumption, and enhancing test efficiency; S2. Before executing the automated test, configure the level of the test script; It should be noted that configuring the level of the test script differentiates its importance and complexity before executing the automated test; providing a basis for subsequent differential retry policies, enabling important test scripts to obtain more sufficient retry opportunities, ensuring the comprehensiveness and accuracy of the test, and optimizing the allocation of test resources; S3. During the execution of the test script, if an exception message is captured, match the exception message with the exception types in the failure retry policy through regular expressions; It should be noted that after capturing the exception message, use regular expressions to match the exception types to accurately identify retryable exceptions; avoiding ineffective retries for non-retryable exceptions, saving resources, and at the same time ensuring that the test script can execute retries according to the policy when encountering retryable exceptions, enhancing test stability; S4. Execute the failure retry policy according to the matching result: If the match is successful, select the corresponding maximum number of retries according to the level of the test script and execute. If the test script is not configured with a level, retry according to the global default number of times; If the match fails, prohibit retries; It should be noted that execute the failure retry policy according to the matching result. If the match is successful, retry according to the test script level or the global default number of times. If the match fails, prohibit retries; realizing reasonable allocation of test resources, ensuring test efficiency and quality, and avoiding resource waste and ineffective retries from interfering with the test process; S5. Add the number of retries during the execution of the test script to the test report; It should be noted that recording the number of retries in the test report provides detailed test execution information, thereby helping testers quickly understand the execution situation and the number of retries of the test script, facilitating the analysis of failure reasons and problem location, and enhancing test maintainability and traceability.

[0024] In this embodiment, the retry policy is centrally managed through a configuration file, making it more convenient to modify the retry policy without hard coding and reducing the maintenance cost. Before execution, the level is configured for the test script, and the exception information is matched in combination with regular expressions. Different retry policies can be executed according to the importance of the test script and the type of exception, improving the flexibility and accuracy of testing. The retry times are recorded in the test report, providing detailed test execution information, which can assist testers in quickly locating and analyzing problems, thereby improving the test quality and efficiency.

[0025] Further, as a refinement and extension of the specific implementation manner of the above embodiment, in order to fully illustrate the specific implementation process in this embodiment, another method for automated test hierarchical retry control is provided. The method includes the following steps: S1. Add a configuration file to the automated test framework and configure the failure retry policy for the test script: Configure the global default number of times; The maximum number of retries for test scripts of each level; The types of exceptions allowed to be retried; The specific steps of step S1 are as follows: S11. Build an automated test framework based on the Python + Pytest + selenium language; S12. Write a failure retry policy file in yaml format and add it to the automated test framework; It should be noted that building an automated test framework based on Python, Pytest, and Selenium can integrate test tools, improve the efficiency of writing and executing test scripts, and ensure the stability and reliability of testing. Writing the failure retry policy file in yaml format makes the file structure clear, highly readable, and convenient for testers to write, read, and maintain the policy configuration, thereby improving the test configuration management efficiency; Exemplarily, use Python, Pytest, and Selenium to build an automated test framework to test an e-commerce website, integrating various test tools to improve the efficiency of writing and executing test scripts; write a YAML format failure retry policy file and add it to the framework. For example, set the global default number of retries to 3 times, and set different retry policies for test scripts of different levels. Level 1 allows 2 retries to handle timeouts and element not found exceptions; Level 2 allows 3 retries to handle assertion errors and value errors; Level 3 allows 1 retry to handle all exceptions; S2. Before executing the automated test, configure the level of the test script; the specific steps of step S2 are as follows: S21. Separate the test cases and test data of the test script under the automated test framework and save the test data to an Excel data table; S22. Determine whether level configurations need to be added to each test case of the test script; If so, proceed to step S23; If not, proceed to step S24; S23. Add level configurations to the test cases in the form of annotations or markings; S24. Add the test cases to the automated test framework; It should be noted that by separating test cases and test data and storing the test data in an Excel sheet, it is convenient to centrally manage, modify, and maintain the test data, avoid modifying the script when modifying the data, and reduce the test maintenance cost; by using annotations or markings to configure levels for test cases, different test case priorities can be distinguished in a concise manner, so that a differential retry strategy can be executed, enhancing the flexibility and pertinence of testing; Exemplarily, taking the login function of an e-commerce website as an example, we separate the test cases and test data and save the test data in an Excel file for convenient centralized management and maintenance; for example, the Excel file contains the login information and expected results of multiple users; before executing the automated test, add level configurations to each test case and clearly define its priority in the form of annotations or markings for subsequent execution of a differential retry strategy; for example, the test case for successful login is marked as level 1, and the test case for failed login is marked as level 2; S3. During the execution of the test script, if exception information is captured, match the exception information with the exception types in the failure retry strategy using regular expressions; the specific steps of step S3 are as follows: S31. Under the automated test framework, import the test data for the test case and execute it; S32. Determine whether exception information is captured during the execution of the test case; If not, wait for a set period of time and return to step S32; If so, proceed to step S33; S33. Convert the exception information into a string form and use regular expressions to match the exception types allowed for retry in the failure retry strategy; It should be noted that by importing the test data and then executing the test case, judging and capturing the exception information, and using regular expressions to match the exception types, accurate identification of retryable exceptions can be achieved, ensuring the accuracy and stability of testing; by judging again whether exception information is captured after waiting for a set period of time, misjudgment caused by temporary system failures or network fluctuations can be prevented, providing a more stable test environment and improving the reliability of test results; Exemplarily, when executing the test script, the automated test framework imports the test data and starts to execute the test cases; if exception information is captured during the execution, the exception information is matched with the exception types defined in the failure retry policy using regular expressions to identify retryable exceptions; for example, if a timeout exception is encountered during the login function test, the system will identify that this exception belongs to the type that allows retries; to prevent misjudgment caused by temporary system failures or network fluctuations, the system will wait for a set period of time when no exception information is captured and then determine again whether exception information is captured; S4. Execute the failure retry policy according to the matching result: If the match is successful, select the corresponding maximum number of retries according to the level of the test script and execute. If the test script is not configured with a level, retry according to the global default number of times; If the match fails, retry is prohibited; The specific steps of step S4 are as follows: S41. Judge the matching result of the regular expression; If the match fails, mark the test item of the test case as test failed and enter step S46; If the match is successful, enter step S42; S42. Judge whether the test case is configured with a level; If so, use the maximum number of retries corresponding to the level as the number of retries that can be performed; If not, use the global default number of times as the number of retries that can be performed; S43. Execute a retry for the corresponding test item of the test case and increment the number of retries by 1; S44. Judge whether the corresponding test item of the test case is executed successfully; If so, enter step S46; If not, enter step S45; S45. Judge whether the number of retries exceeds the number of retries that can be performed; If so, mark the test item of the test case as test failed and enter step S5; If not, return to step S43; S46. Judge whether the current test case is executed completely; If so, enter step S5; If not, continue to execute the test case and return to step S'3'; It should be noted that whether to retry is determined by judging the matching result. If the matching fails, the test is directly marked as failed to avoid wasting resources due to ineffective retries. If the matching succeeds, the retry is performed according to the level or the global default number of times to improve resource utilization. During the retry process, the execution status of the test case and the number of retries are dynamically judged, and the failure is marked in time or the execution is continued to ensure the coherence and integrity of the test, and improve the test efficiency and quality. After the execution is completed, the next step of test report generation is entered. If the execution is not completed, it returns to continue the execution, forming a complete closed-loop test process to ensure the automation and standardization of the test process; Exemplarily, when the exception information matches successfully with the exception types allowing retries, the corresponding number of retries is selected according to the level of the test script and executed. If the test script is not configured with a level, the retry is performed according to the global default number of times. For example, if the login success test case (level 1) encounters a timeout exception, the system will retry 2 times. If the exception information fails to match, the retry is prohibited and the test item is directly marked as failed. During the retry process, the system dynamically judges the execution status of the test case and the number of retries. If the test case is executed successfully during the retry process, the retry stops. If the number of retries exceeds the maximum allowed number of retries, the system marks the test item as failed. For example, if the login failure test case (level 2) is still not successful after retrying 3 times, the system marks it as failed. After the entire test case is executed, the system enters the next step of generating a test report; S5. Add the number of retries during the execution of the test script to the test report; Exemplarily, after the test execution is completed, the system generates a detailed test report, which includes the execution results of each test script and the number of retries. For example, the test report may show that a certain test case is executed successfully without retries, and another test case is executed failed and retried 3 times, and finally is marked as failed due to the exception still not being resolved. The test report provides comprehensive test execution information for testers and can quickly locate and analyze problems.

[0026] In an embodiment of the present invention, different from the above embodiment, the following steps are further included: Create a policy object for the failure retry policy file through a global variable under the automated test framework and integrate a file system monitoring component; During the execution of the test script, the file system monitoring component is used to monitor the modification events of the yaml-format failure retry policy file in real time; When the modification event is detected, automatic configuration parsing is performed, and the updated failure retry policy file is reloaded using the PyYAML library and replaces the old failure retry policy in the memory; It should be noted that a policy object is created for the failure retry policy file through global variables, and a file system monitoring component is integrated to achieve real-time monitoring of the policy file, ensuring that the test process can obtain the latest policy at any time; the updated policy file can be automatically loaded without restarting the process, greatly reducing test interruptions and delays caused by policy adjustments, improving test efficiency and flexibility, and adapting to changing test requirements; finally, the policy object is managed through global variables to ensure that the test script directly reads the latest configuration during execution without restarting the process.

[0027] In an embodiment of the present invention, different from the above embodiment, The failure retry policy further includes the following content: Calibrate the retry count for the first exception type; Increase the retry count for the second exception type; During the execution of the test script, If the exception information matches the first exception type, use the calibrated retry count to overwrite the retryable count; If the exception information matches the second exception type, then add the retryable count and the increased retry count; It should be noted that by flexibly setting the retry logic for different exception types, setting the calibrated retry count for common exceptions for quick processing, and increasing the retry count for complex exceptions to improve the solution probability, the test adaptability and flexibility are enhanced; when matching different exception types, execute the corresponding retry logic according to the settings, making the test retry policy more in line with actual needs, optimizing resource allocation, and improving test efficiency and success rate.

[0028] It should be understood that the magnitudes of the sequence numbers of the steps in the above embodiments do not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.

[0029] As Figure 2 shown, the following is an embodiment of the automated test hierarchical retry control system provided by the embodiments of the present disclosure. This system belongs to the same inventive concept as the automated test hierarchical retry control method of the above embodiments. For the details not described in detail in the embodiment of the automated test hierarchical retry control system, reference can be made to the embodiments of the above automated test hierarchical retry control method.

[0030] The system includes: A retry policy configuration module, used to add a configuration file in the automated test framework and configure the failure retry policy for the test script: Configure the global default count; The maximum retry count for each level of test script; The exception types allowed for retry; A script level configuration module, which is used to configure the level of the test script before performing automated testing; A failure retry policy matching module, which is used to match the exception information with the exception type in the failure retry policy through regular expressions during the execution of the test script if exception information is captured; A failure retry policy execution module, which is used to execute the failure retry policy according to the matching result: If the match is successful, select the corresponding maximum number of retries according to the level of the test script and execute it. If the test script does not have a configured level, perform the retry according to the global default number; If the match fails, retry is prohibited; A test report generation module, which is used to add the number of retries during the execution of the test script to the test report.

[0031] Through the interaction and cooperation of the retry policy configuration module, script level configuration module, failure retry policy matching module, failure retry policy execution module, and test report generation module in this embodiment, the retry policy is adjusted according to different exception types and the priority of the test script, avoiding resource waste.

[0032] The automated test hierarchical retry control method provided by the embodiments of the present application can be applied to an electronic device. Those skilled in the art can understand that the structure of the electronic device involved in the embodiments of the present invention does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements. In the embodiments of the present invention, the electronic device includes, but is not limited to, a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the embodiments of the present application described herein and / or claimed.

[0033] The electronic device may include a processor, an external memory interface, an internal memory, a universal serial bus (USB) interface, a charging management module, a power management module, a battery, a wireless communication module, an audio module, a speaker, a microphone, a sensor module, a key, a camera, a display screen, and a SIM card interface, etc.

[0034] It can be understood that the structure illustrated in the embodiments of the present application does not constitute a specific limitation on the electronic device. In other embodiments of the present application, the electronic device may include more or fewer components than those illustrated, or combine certain components, or split certain components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0035] The processor may include one or more processing units. For example, the processor may include a central processing unit (CPU), an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent devices or integrated in one or more processors.

[0036] Among them, the processor may be the nerve center and command center of the electronic device. The controller may generate operation control signals according to the instruction operation code and timing signals to complete the control of fetching and executing instructions.

[0037] A memory may also be provided in the processor for storing instructions and data. In some embodiments, the memory in the processor is a cache memory. This memory may store the instructions or data that the processor has just used or recycled. If the processor needs to use the instruction or data again, it can directly call it from this memory. This avoids repeated accesses, reduces the waiting time of the processor, and thus improves the system efficiency.

[0038] The above-mentioned electronic device implements the technical solution of adding a configuration file to the automated test framework in the automated test hierarchical retry control method of the present application, configuring a failure retry policy for test scripts: configuring the global default number of times; the maximum number of retries for test scripts of each level; the types of exceptions allowed for retries; before executing the automated test, configuring the level of the test script; during the execution of the test script, if exception information is captured, using regular expressions to match the exception information with the types of exceptions in the failure retry policy; according to the matching result, executing the failure retry policy: if the match is successful, selecting the corresponding maximum number of retries according to the level of the test script and executing it. If the test script does not have a configured level, the retry is executed according to the global default number of times; if the match fails, retries are prohibited; adding the number of retries during the execution of the test script to the test report, achieving the beneficial effects of being able to flexibly adjust the retry policy according to different exception types and the priorities of test scripts, avoiding waste of resources, and at the same time ensuring the real-time nature of the test policy through dynamic updates to adapt to the rapid changes in the software development process.

[0039] In the storage medium provided by the present application, there is a program product capable of implementing the automated test hierarchical retry control method.

[0040] The automated test hierarchical retry control method includes: adding a configuration file to the automated test framework, configuring a failure retry policy for test scripts: configuring the global default number of times; the maximum number of retries for test scripts of each level; the types of exceptions allowed for retries; before executing the automated test, configuring the level of the test script; during the execution of the test script, if exception information is captured, using regular expressions to match the exception information with the types of exceptions in the failure retry policy; according to the matching result, executing the failure retry policy: if the match is successful, selecting the corresponding maximum number of retries according to the level of the test script and executing it. If the test script does not have a configured level, the retry is executed according to the global default number of times; if the match fails, retries are prohibited; adding the number of retries during the execution of the test script to the test report.

[0041] In some possible implementation manners, the automated test hierarchical retry control method of the present disclosure may be implemented in the form of a program product, which includes program code. When the program product runs on a terminal device, the program code is used to cause the terminal device to execute the steps according to various exemplary embodiments of the present disclosure described in the "Exemplary Method" section above of this specification.

[0042] The storage medium of the present disclosure may adopt any combination of one or more readable media. The readable media may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0043] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An automated test hierarchical retry control method, characterized in that: It includes the following steps: S1. Add a configuration file to the automated test framework to configure the failure retry policy for the test script: Configure the global default number of times; The maximum number of retries for test scripts of each level; The types of exceptions allowed for retries; S2. Before executing the automated test, configure the level of the test script; S3. During the execution of the test script, if an exception message is captured, match the exception message with the exception types in the failure retry policy through regular expressions; S4. Execute the failure retry policy according to the matching result: If the match is successful, select the corresponding maximum number of retries according to the level of the test script and execute. If the test script is not configured with a level, execute the retry according to the global default number of times; If the match fails, retry is prohibited; S5. Add the number of retries during the execution of the test script to the test report.

2. The automated test hierarchical retry control method according to claim 1, wherein The specific steps of step S1 are as follows: S11. Build an automated test framework based on the Python + Pytest + selenium language; S12. Write a failure retry policy file in yaml format and add it to the automated test framework.

3. The automated test hierarchical retry control method according to claim 2, wherein, The specific steps of step S2 are as follows: S21. Under the automated test framework, separate the test cases and test data of the test script, and save the test data to an Excel data table; S22. Determine whether each test case of the test script needs to add a level configuration; If so, go to step S23; If not, go to step S24; S23. Add a level configuration to the test case in the form of annotations or markings; S24. Add the test case to the automated test framework.

4. The automated test hierarchical retry control method according to claim 3, wherein The specific steps of step S3 are as follows: S31. Under the automated test framework, import test data for the test case and execute it; S32. Determine whether an exception message is captured during the execution of the test case; If not, wait for the set time period and return to step S32; If so, go to step S33; S33. Convert the exception message into a string form and match it with the exception types allowed for retries in the failure retry policy using regular expressions.

5. The automated test hierarchical retry control method according to claim 4, wherein The specific steps of step S4 are as follows: S41. Judge the matching result of the regular expression; If the match fails, mark the test item of the test case as a test failure and go to step S46; If the match is successful, go to step S42; S42. Determine whether the test case is configured with a level; If so, use the maximum number of retries for the corresponding level as the number of retries allowed; If not, use the global default number of times as the number of retries allowed; S43. Execute the retry for the corresponding test item of the test case and increment the number of retries by 1; S44. Determine whether the corresponding test item of the test case is executed successfully; If so, go to step S46; If not, go to step S45; S45. Determine whether the number of retries exceeds the number of retries allowed; If so, mark the test item of the test case as a test failure and go to step S5; If not, return to step S43; S46. Determine whether the current test case has been executed; If so, go to step S5; If not, continue to execute the test case and return to step S3.

6. The automated test hierarchical retry control method according to claim 2, wherein It also includes the following steps: Create a policy object for the failure retry policy file through global variables under the automated test framework, and integrate the file system monitoring component; During the execution of the test script, the file system monitoring component is used to monitor the modification events of the failure retry policy file in yaml format in real time; When the modification event is detected, automatic configuration parsing is performed, and the updated failure retry policy file is reloaded using the PyYAML library, and the old failure retry policy in memory is replaced.

7. The automated test hierarchical retry control method according to claim 5, wherein The failure retry policy also includes the following content: Calibrate the retry times for the first exception type; Increase the retry times for the second exception type; During the execution of the test script, If the exception information matches the first exception type, use the calibrated retry times to overwrite the retryable times; If the exception information matches the second exception type, then add the retryable times and the increased retry times.

8. An automated test hierarchical retry control system, characterized in that Include: A retry policy configuration module for adding a configuration file in the automated test framework and configuring the failure retry policy for the test script: Configure the global default times; The maximum retry times for test scripts of each level; The exception types allowed for retry; A script level configuration module for configuring the level of the test script before performing the automated test; A failure retry policy matching module for matching the exception information with the exception types in the failure retry policy through regular expressions during the execution of the test script if an exception information is captured; A failure retry policy execution module for executing the failure retry policy according to the matching result: If the match is successful, select the corresponding maximum retry times according to the level of the test script and execute. If the test script is not configured with a level, retry according to the global default times; If the match fails, retry is prohibited; A test report generation module for adding the retry times during the execution of the test script to the test report.

9. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that when the processor executes the program, the steps of the automated test hierarchical retry control method according to any one of claims 1 to 7 are implemented.

10. A storage medium, on which a computer program is stored, characterized in that, When the computer program is executed by the processor, the steps of the automated test hierarchical retry control method according to any one of claims 1 to 7 are implemented.

Images