Automatic testing method and device, medium and program product

Abstract

The invention discloses an automatic testing method and device, a medium and a program product. The method is suitable for the technical field of cloud computing, and comprises the following steps: monitoring an automatic test environment and test data in a process of executing an automatic test, and executing a repair operation when monitoring an exception; if the target automation case which fails to be tested exists, performing historical anomaly detection according to the execution failure time of the target automation case; and when the historical abnormal record is detected, re-executing the target automation case. By adopting the technical scheme, the test environment and the test data can be monitored in the automatic test process, and when abnormity is monitored, automatic repair is carried out, so that the false alarm rate of the automatic test is greatly reduced, and the test efficiency is effectively improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of cloud computing, and particularly relates to an automatic testing method, device, medium and program product. BACKGROUND

[0002] Automatic testing refers to a process of simulating user operation or system interaction through a predefined script, tool or framework, automatically executing test cases, verifying software behavior, and generating a result report. In automatic testing, a false positive refers to a situation where the automatic test case report indicates that there is a problem, but the function of the system under test is normal and there is no real problem, which will interfere with the test process and reduce the team's trust in automatic testing. Automatic testing false positives are generally caused by environmental problems, test data problems or test scripts.

[0003] In the prior art, a pre-established error code library and a non-defect knowledge base are generally used to classify and analyze test results, thereby identifying false positive problems.

[0004] Although the prior art can identify false positive information, it cannot actually solve the problem of automatic test failure caused by external factors during the automatic test execution process, which greatly affects the efficiency and quality of automatic testing. SUMMARY

[0005] The present application provides an automatic testing method, device, medium and program product, which can monitor the test environment and test data during the automatic testing process, and perform self-repair when an anomaly is detected, thereby greatly reducing the false positive rate of automatic testing and effectively improving the test efficiency.

[0006] According to an aspect of the present application, an automatic testing method is provided, comprising:

[0007] During the execution of the automatic testing, the automatic testing environment and test data are monitored, and a repair operation is performed when an anomaly is detected;

[0008] If there is a target automatic case that fails, historical anomaly detection is performed according to the execution failure time of the target automatic case;

[0009] When a historical anomaly record is detected, the target automatic case is re-executed.

[0010] Optionally, during the execution of the automatic testing, the automatic testing environment and test data are monitored, and a repair operation is performed when an anomaly is detected, comprising:

[0011] In the process of executing the automated test, the availability of the automated test environment is detected by the target detection script at a regular time;

[0012] If the automated test environment is detected as unavailable, the target environment repair script is called to repair the automated test environment according to the detection return of the automated test environment, and the abnormal information of the automated test environment is recorded in the database.

[0013] The advantage of such a setting is that the target detection script can quickly detect the problem of unavailable environment, avoid batch test task failure caused by environment abnormality, reduce invalid test time, and once the abnormality is detected, the system can directly call the environment repair script to quickly restore the availability of the environment and ensure the test process to proceed as planned.

[0014] Optionally, in the process of executing the automated test, the automated test environment and the test data are monitored, and when an abnormality is detected, a repair operation is performed, including:

[0015] In the process of executing the automated test, the validity of the test data is detected at a regular time according to the target check rule of the test data.

[0016] If the test data is detected as invalid data, the target correction rule is determined according to the detection result of the test data, the test data is modified by using the target correction rule, and the abnormal information of the test data is recorded in the database.

[0017] The advantage of such a setting is that it can reduce test interruption and false positives caused by data problems, ensure smooth automated test process, improve the credibility of test results, and solve the problem of resource waste and low test efficiency caused by manual inspection and maintenance of data in the prior art.

[0018] Optionally, if there is a target automated case that fails the test, historical abnormality detection is performed according to the execution failure time of the target automated case, including:

[0019] If there is a target automated case that fails the test, the target time interval is determined according to the execution failure time of the target automated case.

[0020] According to the target time interval, the abnormal information of the automated test environment and the abnormal information of the test data are detected in the database, and when any abnormal information recorded in the target time interval is detected, it is determined that the historical abnormal record related to the target automated case is detected.

[0021] The advantage of this setup is that by detecting abnormal information within the target time interval, it is possible to determine to some extent whether the failure of the target automated test case is due to abnormal automated testing environment or abnormal test data. This can avoid false alarms and ineffective modifications to the automated test case scripts, thereby improving detection efficiency and the efficiency of automated testing.

[0022] Optionally, after re-executing the target automation case when historical anomalies are detected, the following may also be included:

[0023] A test report is generated based on the initial test results of the automated tests and the test results after each target automated test case is re-executed.

[0024] The advantage of this setup is that it provides accurate test results for test analysis, avoids false alarms in automated testing caused by external factors, thereby improving the accuracy of test reports and enhancing the efficiency and reliability of automated testing.

[0025] Optionally, during the execution of automated testing, the automated testing environment and test data are monitored, and before performing remedial operations upon detecting an anomaly, the following steps are also included:

[0026] Obtain the environment information of the automated testing environment, and based on the environment information, determine the target detection scripts and environment repair script sets that match the current automated testing system.

[0027] The advantage of this setup is that by matching the detection scripts with the environmental information, the detection logic can be made to fit the characteristics of the current environment, which can effectively reduce false alarms and false negatives. The detection scripts matched with the environmental information will prioritize detecting key indicators in the current scenario, avoiding indiscriminate detection that consumes too many system resources.

[0028] Optionally, during the execution of automated testing, the automated testing environment and test data are monitored, and before performing remedial operations upon detecting an anomaly, the following steps are also included:

[0029] Obtain the environment information and test data types of the automated testing environment, and determine the target inspection rules and correction rule set that match the current automated testing system based on the environment information and test data types of the automated testing environment.

[0030] The advantage of this setup is that the inspection and correction rules are selected based on the test environment and the type of test data, which can effectively improve the detection accuracy and repair efficiency of test data, avoid misjudgment of test data, and ensure that problem identification is more in line with the actual scenario.

[0031] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0032] At least one processor; and

[0033] A memory communicatively connected to the at least one processor; wherein,

[0034] The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the automated testing method described in any embodiment of the present invention.

[0035] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the automated testing method described in any embodiment of the present invention.

[0036] According to another aspect of the present invention, a computer program product is provided, the computer program product comprising a computer program that, when executed by a processor, implements the automated testing method described in any embodiment of the present invention.

[0037] The technical solution of this invention, by monitoring the automated testing environment and test data during the execution of automated testing and performing repair operations when anomalies are detected, can quickly identify unusable environments and invalid test data, avoiding batch failures of test tasks due to environmental anomalies or test data, reducing invalid testing time, and ensuring the normal operation of subsequent tests through automatic repair operations. By performing historical anomaly detection based on the execution failure time of target automated cases if there are failed tests, it can determine to some extent whether the failure of target automated cases is caused by abnormal automated testing environment or test data, avoiding false positives and invalid modifications to automated case scripts, improving detection efficiency and automated testing efficiency. By re-executing target automated cases when historical anomaly records are detected, the influence of automated testing environment factors or test data factors on the test results of target automated cases can be overcome, avoiding test failures caused by automated testing environment factors or test data factors.

[0038] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 This is a flowchart of an automated testing method provided according to Embodiment 1 of the present invention;

[0041] Figure 2 This is a schematic diagram of the structure of an automated testing environment provided according to an embodiment of the present invention;

[0042] Figure 3 This is a flowchart of another automated testing method provided according to Embodiment 2 of the present invention;

[0043] Figure 4 This is a schematic diagram of the structure of an automated testing device according to Embodiment 3 of the present invention;

[0044] Figure 5 This is a schematic diagram of the structure of an electronic device that implements the automated testing method of this invention. Detailed Implementation

[0045] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0046] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0047] Example 1

[0048] Figure 1This is a flowchart of an automated testing method provided in Embodiment 1 of the present invention. This embodiment is applicable to situations where abnormalities are monitored and automatically repaired during automated testing, and abnormal cases are repeatedly executed. This method can be executed by an automated testing device, which can be implemented in hardware and / or software, and is generally configured in a computer or processor with data processing capabilities. Figure 1 As shown, the method includes:

[0049] S110. During the execution of automated testing, the automated testing environment and test data are monitored, and repair operations are performed when an anomaly is detected.

[0050] Optionally, an automated testing environment can refer to a complete ecosystem that runs the software under test and provides all the hardware, software, network configurations, data, and dependencies required for its normal operation; test data can refer to various types of data used in the software testing process to verify the system's functions, performance, security, and other characteristics. Test data can simulate various inputs in real business scenarios, and by observing the system's response to this data, it can be determined whether the system meets the expected design.

[0051] Optionally, automated testing can be executed within an automated testing system. This application proposes two parallel systems based on existing automated testing systems: an environment availability detection and repair system, and a data inspection and correction system. The environment availability detection and repair system periodically detects the availability of the automated testing environment. When the environment is unavailable, it calls the corresponding environment repair script, such as restarting the service, based on the actual detection response, to restore the automated testing environment and resolve false alarms caused by the unavailability of the automated testing environment. Simultaneously, it records the environment anomaly information to the database. The data inspection and correction system periodically checks the validity of test data according to configured test data inspection rules. If the test data does not meet expectations, it modifies the problematic test data to the expected data format according to configured correction rules, resolving false alarms caused by problematic test data. Simultaneously, it records the test data anomaly information to the database.

[0052] Figure 2 This is a schematic diagram of an optional automated testing environment. (For example...) Figure 2 As shown, the automated testing system, the environmental availability detection and repair system, and the data inspection and correction system are connected to the environment under test and the database (DB), respectively.

[0053] During the execution of automated testing, monitoring the automated testing environment and test data, and before performing remedial operations upon detecting anomalies, may also include:

[0054] Obtain the environment information of the automated testing environment, and based on the environment information, determine the target detection scripts and environment repair script sets that match the current automated testing system.

[0055] Optionally, the environment information of the automated testing environment can refer to all the environment attributes and configuration data on which the automated testing system depends for operation. It can include one or more of the following: hardware layer information, software layer information, configuration layer information, and dependency layer information. There is no limitation on this. For example, it can include network configuration information such as IP address and domain name.

[0056] Optionally, in the environment availability detection and repair system, a variety of different detection scripts and environment repair scripts can be pre-set. When the environment availability detection and repair system is applied to different automated testing environments, the detection scripts and environment repair scripts can be configured according to the environment information, so as to realize the anomaly detection and repair of the automated testing environment by using the configured scripts.

[0057] Optionally, a target detection script can refer to a script used to detect whether there are any anomalies in the automated testing environment. Its function is to actively detect whether the environment meets the preset standards. Different environment information corresponds to different detection logic. For example, a detection script for the first version of the database will check whether window functions are supported, while a detection script for the second version of the database will not check this item.

[0058] Optionally, the environment remediation script set can refer to a collection of scripts used to remediate environmental anomalies, with each script corresponding to a solution for a specific environmental problem. The environment remediation script set includes at least one environment remediation script that matches the current automated testing system.

[0059] Optionally, the system acquires the environment information of the automated testing environment and, based on this information, determines a set of target detection scripts and environment repair scripts that match the current automated testing system. These may include:

[0060] Obtain the environment information of the automated testing environment, and construct an environment profile of the automated testing environment based on the environment information; the environment profile is used to describe the various dimensions of information of the automated testing environment;

[0061] Based on the environment profile of the automated testing environment, target detection scripts and environment repair script sets that match the current automated testing system are mapped and obtained from the script library; each script in the script library includes preset applicable environment tags;

[0062] When changes in environmental information are detected, the environmental profile is automatically updated and a matching process is triggered. Based on the matching results, the target detection script and the environmental repair script set are updated.

[0063] The advantage of this setup is that by matching the detection scripts with the environmental information, the detection logic can be made to fit the characteristics of the current environment, which can effectively reduce false alarms and false negatives. The detection scripts matched with the environmental information will prioritize detecting key indicators in the current scenario, avoiding indiscriminate detection that consumes too many system resources.

[0064] During the execution of automated testing, monitoring the automated testing environment and test data, and before performing remedial operations upon detecting anomalies, may also include:

[0065] Obtain the environment information and test data types of the automated testing environment, and determine the target inspection rules and correction rule set that match the current automated testing system based on the environment information and test data types of the automated testing environment.

[0066] Optionally, the test data type can refer to the classification attributes of input, output, and intermediate data during the automated testing process, based on dimensions such as data purpose, format, and sensitivity.

[0067] Optionally, in the data inspection and correction system, various inspection rules and correction rules can be pre-set. The detection rules and correction rules can be configured according to the environment information of the automated testing environment and the type of test data used in the current automated testing, so as to realize the detection and repair of test data anomalies using the configured inspection rules and correction rules.

[0068] Optionally, the test data type can refer to the classification attributes of input data, intermediate data, and expected result data used in the automated testing process. The target inspection rule can be used to verify the validity of the test data. The correction rule set can refer to a set of preset logic used to automatically repair problems when the inspection rule determines that there are abnormalities in the environment or data. The correction rule set can include at least one correction rule, and the correction rule can be matched with the abnormal cause of the test data.

[0069] Optionally, when the automated testing environment and / or the type of test data change, the target inspection rules and the set of correction rules can be automatically updated according to the changes without refactoring the overall testing framework, and can be quickly adapted to the current testing scenario.

[0070] The advantage of this setup is that the inspection and correction rules are selected based on the test environment and the type of test data, which can effectively improve the detection accuracy and repair efficiency of test data, avoid misjudgment of test data, and ensure that problem identification is more in line with the actual scenario.

[0071] S120. If there are target automation cases that have failed to be tested, then perform historical anomaly detection based on the execution failure time of the target automation cases.

[0072] Optionally, automated test cases refer to pre-set test cases that are executed automatically through scripts, tools, etc., to verify the functionality, performance, compatibility, and other characteristics of software. In an automated test process, there may be a batch of automated test cases, and each automated test case is tested in sequence. After each round of automated testing, the automated testing system can record the test results of each automated test case and the execution success or failure time in the database. The test results may include records of whether the automated test case was successful or failed.

[0073] If there are failed target automation cases, historical anomaly detection will be performed based on the execution failure time of the target automation cases, which may include:

[0074] If there are target automation cases that fail to pass the test, then the target time interval is determined based on the execution failure time of the target automation cases.

[0075] Based on the target time interval, detect abnormal information in the automated testing environment and test data in the database. When any abnormal information recorded within the target time interval is detected, determine that a historical abnormal record related to the target automated case has been detected.

[0076] Optionally, after completing a round of automated testing, if at least one target automated test case fails in this round of automated testing, the execution failure time recorded in the target automated test case is retrieved from the database, and it is determined whether abnormal information of the automated testing environment and / or abnormal information of the test data are recorded within the target time interval where the execution failure time is located.

[0077] Optionally, the target time interval can be determined based on the execution failure time of the target automated case and the preset interval size. For example, the interval size can be set to 5 minutes, and the 5 minutes before and after the execution failure time can be defined as the target time interval. Historical anomaly detection is performed by judging whether abnormal information is recorded in the database within the target time interval.

[0078] The advantage of this setup is that by detecting abnormal information within the target time interval, it is possible to determine to some extent whether the failure of the target automated test case is due to abnormal automated testing environment or abnormal test data. This can avoid false alarms and ineffective modifications to the automated test case scripts, thereby improving detection efficiency and the efficiency of automated testing.

[0079] S130. When a historical anomaly record is detected, re-execute the target automation case.

[0080] Optionally, if abnormal information is recorded in the database within the target time interval, it is determined that the test failure was caused by factors of the automated testing environment and / or test data. If test failures caused by factors of the automated testing environment or test data are recorded in the test report, there will be false alarms, thereby reducing the testing efficiency and analysis quality of automated testing.

[0081] Optionally, if a historical anomaly record is detected, it is determined that the target automated case may have failed due to factors in the automated testing environment or test data. Since the anomaly has been actively repaired after it was detected in step S110 of this invention, the target automated case can be executed again to overcome the influence of factors in the automated testing environment or test data on the test results of the target automated case.

[0082] The technical solution of this invention, by monitoring the automated testing environment and test data during the execution of automated testing and performing repair operations when anomalies are detected, can quickly identify unusable environments and invalid test data, avoiding batch failures of test tasks due to environmental anomalies or test data, reducing invalid testing time, and ensuring the normal operation of subsequent tests through automatic repair operations. By performing historical anomaly detection based on the execution failure time of target automated cases if there are failed tests, it can determine to some extent whether the failure of target automated cases is caused by abnormal automated testing environment or test data, avoiding false positives and invalid modifications to automated case scripts, improving detection efficiency and automated testing efficiency. By re-executing target automated cases when historical anomaly records are detected, the influence of automated testing environment factors or test data factors on the test results of target automated cases can be overcome, avoiding test failures caused by automated testing environment factors or test data factors.

[0083] Example 2

[0084] Figure 3 This is a flowchart of an automated testing method provided in Embodiment 2 of the present invention. This embodiment specifically illustrates the automated testing method based on the above embodiments. Figure 3 As shown, the method includes:

[0085] S210. Obtain the environment information of the automated testing environment, and based on the environment information of the automated testing environment, determine the target detection script and environment repair script set that match the current automated testing system.

[0086] S220. Obtain the environment information and test data types of the automated testing environment, and determine the target inspection rules and correction rule set that match the current automated testing system based on the environment information and test data types of the automated testing environment.

[0087] It is understood that steps S210 and S220 can actually be performed in any order, either in parallel or in any other order; this is only an example.

[0088] S230. During the execution of automated testing, the automated testing environment and test data are monitored, and when an anomaly is detected, repair operations are performed.

[0089] During the execution of automated testing, monitoring the automated testing environment and test data, and performing remedial operations when anomalies are detected, may include:

[0090] During the execution of automated testing, target probing scripts are used to periodically probe the availability of the automated testing environment;

[0091] If the automated testing environment is detected as unavailable, the target environment repair script will be invoked to repair the automated testing environment based on the detection results, and the abnormal information of the automated testing environment will be recorded in the database.

[0092] Optionally, the automatic probing cycle of the automated testing environment can be preset, for example, to initiate an availability probe on the environment under test every 60 seconds. Availability probes can include HTTP 200 checks, API endpoint checks, etc.

[0093] Optionally, the probe return can refer to the result data about the environment status returned by the target probe script after periodically probing the availability of the automated test environment. It can be used to determine whether the automated test environment is normal. The probe return may include, but is not limited to, information such as the current status of the environment and the specific manifestations when an anomaly occurs. The current status of the environment can be used to describe the availability of the environment, and the specific manifestations when an anomaly occurs can be used to describe the root cause of the detected anomaly, such as error codes, log fragments, etc.

[0094] Optionally, the anomaly information can refer to the detailed data extracted from the detection return when the automated testing environment is detected as unavailable, which describes the environment anomaly. This may include, but is not limited to, information such as anomaly type, occurrence time, and scope of impact, such as cluster name, domain name, IP address, and time point when the anomaly was checked.

[0095] The advantages of this setup are: the target detection script can quickly detect environmental unavailability issues by periodically probing, avoiding batch failures of test tasks due to environmental anomalies, reducing invalid test time, and once an anomaly is detected, the system can directly call the environment repair script to quickly restore environmental availability without waiting for testers to investigate, ensuring that the test process proceeds as planned.

[0096] During the execution of automated testing, monitoring the automated testing environment and test data, and performing remedial operations when anomalies are detected, may include:

[0097] During the execution of automated testing, the validity of the test data is checked periodically according to the target checking rules for the test data.

[0098] If invalid test data is detected, a target correction rule is determined based on the test data detection results. The test data is then modified using the target correction rule, and the abnormal information of the test data is recorded in the database.

[0099] Optionally, the testing period for test data can be preset, for example, the validity of the test data can be checked every 20 minutes. The testing period for test data can be the same as or different from the detection period of the automated testing environment. There is no limitation on this. The validity check of test data can be, for example, checking the correlation between data from different tables. When the correlation between data from at least two tables does not meet expectations, the test data is determined to be invalid. This is only an example and does not limit the specific rules for checking test data.

[0100] Optionally, validity testing can include three levels of testing: format validity, logical validity, and scenario validity. The test results can include conclusions on whether the data is valid, data rules violated by invalid data, and location information of invalid data.

[0101] Optionally, the abnormal information of the test data may include, but is not limited to, the test data detection results, detection time, associated test case ID, corrective measures and results, etc., but there is no limitation on the abnormal information of the test data.

[0102] The advantages of this setup are: it reduces test interruptions and false alarms caused by data issues, ensures a smooth automated testing process, improves the reliability of test results, and solves the problems of resource waste and low testing efficiency caused by manual data inspection and maintenance in existing technologies.

[0103] S240. If there is a target automation case that failed to pass the test, then determine the target time interval based on the execution failure time of the target automation case.

[0104] S250. Based on the target time interval, detect abnormal information of the automated testing environment and abnormal information of the test data in the database, and when any abnormal information recorded within the target time interval is detected, determine that a historical abnormal record related to the target automated case has been detected.

[0105] S260. When a historical anomaly record is detected, re-execute the target automation case.

[0106] S270. Generate a test report based on the initial test results of the automated test and the test results after re-execution of each target automated case.

[0107] Optionally, the initial test results can be corrected based on the test results after the target automation case is re-executed, and a test report can be generated based on the corrected test results. The specific correction methods are as follows: replace the initial test results of the corresponding target automation case with the test results of the target automation case that was successfully executed after re-execution; or, directly merge the two test results, and retain both test results when the target automation case has been executed twice.

[0108] The advantage of this setup is that it provides accurate test results for test analysis, avoids false alarms in automated testing caused by external factors, thereby improving the accuracy of test reports and enhancing the efficiency and reliability of automated testing.

[0109] The technical solution of this invention, by monitoring the automated testing environment and test data during the execution of automated testing and performing repair operations when anomalies are detected, can quickly identify unusable environments and invalid test data, avoiding batch failures of test tasks due to environmental anomalies or test data, reducing invalid testing time, and ensuring the normal operation of subsequent tests through automatic repair operations. By performing historical anomaly detection based on the execution failure time of target automated cases if there are failed tests, it can determine to some extent whether the failure of target automated cases is caused by abnormal automated testing environment or test data, avoiding false positives and invalid modifications to automated case scripts, improving detection efficiency and automated testing efficiency. By re-executing target automated cases when historical anomaly records are detected, the influence of automated testing environment factors or test data factors on the test results of target automated cases can be overcome, avoiding test failures caused by automated testing environment factors or test data factors.

[0110] Example 3

[0111] Figure 4 This is a schematic diagram of an automated testing device provided in Embodiment 3 of the present invention.Figure 4 As shown, the device includes: an anomaly monitoring and repair module 310, a historical anomaly detection module 320, and a repetitive execution module 330.

[0112] The anomaly monitoring and repair module 310 is used to monitor the automated testing environment and test data during the execution of automated testing, and to perform repair operations when an anomaly is detected.

[0113] The historical anomaly detection module 320 is used to perform historical anomaly detection based on the execution failure time of the target automated case if there is a failed test case.

[0114] The re-execution module 330 is used to re-execute the target automation case when a historical anomaly record is detected.

[0115] The technical solution of this invention, by monitoring the automated testing environment and test data during the execution of automated testing and performing repair operations when anomalies are detected, can quickly identify unusable environments and invalid test data, avoiding batch failures of test tasks due to environmental anomalies or test data, reducing invalid testing time, and ensuring the normal operation of subsequent tests through automatic repair operations. By performing historical anomaly detection based on the execution failure time of target automated cases if there are failed tests, it can determine to some extent whether the failure of target automated cases is caused by abnormal automated testing environment or test data, avoiding false positives and invalid modifications to automated case scripts, improving detection efficiency and automated testing efficiency. By re-executing target automated cases when historical anomaly records are detected, the influence of automated testing environment factors or test data factors on the test results of target automated cases can be overcome, avoiding test failures caused by automated testing environment factors or test data factors.

[0116] Based on the above embodiments, the anomaly monitoring and repair module 310 can be specifically used for:

[0117] During the execution of automated testing, target probing scripts are used to periodically probe the availability of the automated testing environment;

[0118] If the automated testing environment is detected as unavailable, the target environment repair script will be invoked to repair the automated testing environment based on the detection results, and the abnormal information of the automated testing environment will be recorded in the database.

[0119] Based on the above embodiments, the anomaly monitoring and repair module 310 can be specifically used for:

[0120] During the execution of automated testing, the validity of the test data is checked periodically according to the target checking rules for the test data.

[0121] If invalid test data is detected, a target correction rule is determined based on the test data detection results. The test data is then modified using the target correction rule, and the abnormal information of the test data is recorded in the database.

[0122] Based on the above embodiments, the historical anomaly detection module 320 can be specifically used for:

[0123] If there are target automation cases that fail to pass the test, then the target time interval is determined based on the execution failure time of the target automation cases.

[0124] Based on the target time interval, detect abnormal information in the automated testing environment and test data in the database. When any abnormal information recorded within the target time interval is detected, determine that a historical abnormal record related to the target automated case has been detected.

[0125] Based on the above embodiments, a report generation module may also be included, used for:

[0126] A test report is generated based on the initial test results of the automated tests and the test results after each target automated test case is re-executed.

[0127] Based on the above embodiments, an environmental detection configuration module may also be included, for:

[0128] Obtain the environment information of the automated testing environment, and based on the environment information, determine the target detection scripts and environment repair script sets that match the current automated testing system.

[0129] Based on the above embodiments, a test data checking configuration module may also be included, used for:

[0130] Obtain the environment information and test data types of the automated testing environment, and determine the target inspection rules and correction rule set that match the current automated testing system based on the environment information and test data types of the automated testing environment.

[0131] The automated testing device provided in the embodiments of the present invention can execute the automated testing method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the method execution.

[0132] Example 4

[0133] Figure 5A schematic diagram of an electronic device 10 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0134] like Figure 5 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0135] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0136] Processor 11 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, central processing unit (CPU), graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as the automated testing methods described in embodiments of the present invention. That is:

[0137] During the execution of automated testing, the automated testing environment and test data are monitored, and repair operations are performed when anomalies are detected.

[0138] If there are target automation cases that failed to pass the test, historical anomaly detection will be performed based on the execution failure time of the target automation cases.

[0139] When a historical anomaly is detected, the target automation case is re-executed.

[0140] In some embodiments, the automated testing method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the automated testing method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the automated testing method by any other suitable means (e.g., by means of firmware).

[0141] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0142] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0143] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0144] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0145] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0146] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0147] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0148] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. An automated testing method, characterized in that, include: During the execution of automated testing, the automated testing environment and test data are monitored, and repair operations are performed when anomalies are detected. If there are target automation cases that failed to pass the test, historical anomaly detection will be performed based on the execution failure time of the target automation cases. When a historical anomaly is detected, the target automation case is re-executed.

2. The method according to claim 1, characterized in that, During automated testing, the automated testing environment and test data are monitored, and remedial actions are performed when anomalies are detected, including: During the execution of automated testing, target probing scripts are used to periodically probe the availability of the automated testing environment; If the automated testing environment is detected as unavailable, the target environment repair script will be invoked to repair the automated testing environment based on the detection results, and the abnormal information of the automated testing environment will be recorded in the database.

3. The method according to claim 1, characterized in that, During automated testing, the automated testing environment and test data are monitored, and remedial actions are performed when anomalies are detected, including: During the execution of automated testing, the validity of the test data is checked periodically according to the target checking rules for the test data. If invalid test data is detected, a target correction rule is determined based on the test data detection results. The test data is then modified using the target correction rule, and the abnormal information of the test data is recorded in the database.

4. The method according to claim 1, characterized in that, If there are failed target automation test cases, historical anomaly detection will be performed based on the execution failure time of the target automation test cases, including: If there are target automation cases that fail to pass the test, then the target time interval is determined based on the execution failure time of the target automation cases. Based on the target time interval, detect abnormal information in the automated testing environment and test data in the database. When any abnormal information recorded within the target time interval is detected, determine that a historical abnormal record related to the target automated case has been detected.

5. The method according to claim 1, characterized in that, After re-executing the target automation case when historical anomalies are detected, the following is also included: A test report is generated based on the initial test results of the automated tests and the test results after each target automated test case is re-executed.

6. The method according to claim 1, characterized in that, During the execution of automated testing, the automated testing environment and test data are monitored, and before remedial actions are performed upon detection of anomalies, the following steps are also included: Obtain the environment information of the automated testing environment, and based on the environment information, determine the target detection scripts and environment repair script sets that match the current automated testing system.

7. The method according to claim 1, characterized in that, During the execution of automated testing, the automated testing environment and test data are monitored, and before remedial actions are performed upon detection of anomalies, the following steps are also included: Obtain the environment information and test data types of the automated testing environment, and determine the target inspection rules and correction rule set that match the current automated testing system based on the environment information and test data types of the automated testing environment.

8. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the automated testing method according to any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the automated testing method according to any one of claims 1-7.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the automated testing method according to any one of claims 1-7.

Images