A regression test control method, apparatus, equipment and storage medium

Abstract

This invention discloses a regression test control method, apparatus, device, and storage medium, relating to the field of computer technology and applied to regression testing tools. The method includes: refreshing a local old configuration file using a received adjusted configuration file, and executing regression tests corresponding to the current test requirements based on the refreshed configuration file; upon detecting the end of the regression test, packaging the regression test results to obtain a test result file carrying a timestamp; parsing the test result file to determine the failed test cases and their failure reasons, classifying the failure reasons corresponding to the failed test cases, filtering the failed test cases based on failure type, and repackaging the filtered test result file to obtain a filtered test result file carrying a new timestamp. The technical solution of this invention enables dynamic control of test case execution, thereby reducing manual intervention and improving verification efficiency.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a regression test control method, apparatus, device, and storage medium. Background Technology

[0002] Currently, verification is a crucial part of the chip development process, determining whether the DUT (Design Under Test) functions correctly. Within the verification process, regression testing of test cases is a vital component throughout the entire verification lifecycle. Initially, to measure the progress of testing, verifiers need to define coverage to observe the extent to which the DUT's functions and scenarios are covered. Regression testing helps developers and testers determine whether previously existing defects have been fixed and prevents new defects from appearing. In the later stages, to achieve coverage convergence as quickly as possible, verifiers will run a large number of test cases multiple times using different seeds. This process also involves frequent regression testing of test cases. This process is mechanical and unavoidable. Verifiers will continuously monitor the running status of test cases and manually select those requiring multiple rounds of regression testing. Furthermore, the numerous regression tests generate numerous log files, which, when accumulated, consume significant memory space and can severely impact server performance during periods of intensive testing.

[0003] Currently, common regression testing tools primarily include functions such as creating test case libraries, manually executing test cases, and generating test reports. Creating a test case library allows verifiers to add test cases to form a library; manually executing test cases allows verifiers to manually select test cases to execute; and generating test reports provides statistics on the execution status of existing test cases, including pass rate and execution progress. However, the execution process supported by these regression testing tools requires manual intervention; that is, verifiers need to dynamically observe and manually select and add test cases. The generated test reports provide a direct representation of the execution status of existing regression test cases, but lack analysis of log files during the execution process. Therefore, it is difficult to effectively assist verifiers in quickly locating the reasons for failed test cases, and there is a lack of categorization of these test cases based on their failure reasons. If multiple rounds of regression operations are continued, new test case results may overwrite old test case results. Summary of the Invention

[0004] In view of this, the purpose of this invention is to provide a regression testing control method, apparatus, device, and storage medium, which can effectively reduce manual intervention and automatically classify the failure reasons of failed test cases, thereby improving verification efficiency. The specific solution is as follows:

[0005] In a first aspect, the present invention discloses a regression test control method, applied to regression testing tools, comprising:

[0006] Receive the modified configuration file imported by the user; the modified configuration file is a configuration file obtained by adjusting the original configuration file according to the current test requirements;

[0007] The local old configuration file is refreshed using the adjusted configuration file, and regression tests corresponding to the current test requirements are executed based on the refreshed configuration file.

[0008] Upon detecting the end of the regression test, the regression test results corresponding to all test cases in the regression test are packaged to obtain a test result file with a timestamp.

[0009] The regression test results corresponding to all test cases in the test result file are parsed to determine the failed test cases that failed the regression test and the failure reasons corresponding to the failed test cases. The failure reasons corresponding to the failed test cases are then classified to determine the failure type corresponding to the failed test cases.

[0010] Based on the failure type, the failed test cases in the test result file are filtered according to the preset filtering rules to obtain a filtered test result file, and the filtered test result file is repackaged to obtain a filtered test result file carrying a new timestamp.

[0011] Optionally, the original configuration file is generated based on basic information extracted from the original test plan; the basic information includes the verification target group to which the test case belongs, the test case name, the number of randomizations, and the test case execution flag.

[0012] Optionally, the step of executing regression tests corresponding to the current test requirements based on the refreshed configuration file includes:

[0013] Extract the target basic information corresponding to each test case from the refreshed configuration file, and store the target basic information in the form of key-value pairs into the pre-built target dictionary;

[0014] The target dictionary is traversed to store the random number of each test case required in the current regression test and the test case execution flag in the pre-built test case table;

[0015] Execute regression tests corresponding to the current test requirements based on the test case table.

[0016] Optionally, the process of classifying the failure reasons corresponding to the failed test cases further includes:

[0017] The first failure test case that appears that is different from other failure types is identified as the header test case of the failure type corresponding to the failure test case.

[0018] Accordingly, the step of filtering the failed test cases in the test result file according to the failure type and a preset filtering rule to obtain a filtered test result file includes:

[0019] The test results file is sequentially checked to determine whether the failed test cases and the target header test cases belong to the same verification target group; the target header test case is a header test case with the same failure type as the failed test cases.

[0020] If the failed test case and the target header test case belong to the same verification target group, then determine whether the failed test case belongs to the test cases that are randomly executed multiple times in regression testing;

[0021] If the failed test case is a test case that is randomly executed multiple times in regression testing, then determine whether the failure reason keyword of the failed test case is consistent with the failure reason keyword of the target header test case.

[0022] If the failure reason keyword of the failed test case is consistent with the failure reason keyword of the target header test case, then the failed test case is filtered.

[0023] If the failure reason keyword of the failed test case is inconsistent with the failure reason keyword of the target header test case, then the failed test case is retained.

[0024] Optionally, after sequentially determining whether the failed test cases and target header test cases in the test result file belong to the same verification target group, the method further includes:

[0025] If the failed test case does not belong to the same verification target group as the target header test case, then the failed test case is retained;

[0026] Accordingly, after determining whether the failed test case belongs to the category of test cases that are randomly executed multiple times in regression testing, the method further includes:

[0027] If the failed test case is not one of the test cases that are randomly executed multiple times in regression testing, then the failed test case is retained.

[0028] Optionally, the regression test control method further includes:

[0029] Obtain historical simulation test data corresponding to the historical regression test, and obtain the latest simulation test data corresponding to the current regression test;

[0030] The historical simulation test data and the latest simulation test data are merged to obtain merged simulation test data, and the merged simulation test data is packaged to obtain a simulation test data file with a timestamp.

[0031] Optionally, obtaining the historical simulation test data corresponding to the historical regression test and obtaining the latest simulation test data corresponding to the current regression test includes:

[0032] When the local preset monitor detects that the current regression test has ended, it will open the local preset first data acquisition interface and second data acquisition interface.

[0033] The first data acquisition interface is used to obtain historical simulation test data corresponding to the historical regression test, and the second data acquisition interface is used to obtain the latest simulation test data corresponding to the current regression test.

[0034] Secondly, this invention discloses a regression testing control device, applied to regression testing tools, comprising:

[0035] The receiving module is used to receive the adjusted configuration file imported by the user; the adjusted configuration file is a configuration file obtained by adjusting the original configuration file according to the current test requirements;

[0036] A refresh module is used to refresh the local old configuration file using the adjusted configuration file;

[0037] The test execution module is used to execute regression tests corresponding to the current test requirements based on the refreshed configuration file.

[0038] The first packaging module is used to package the regression test results corresponding to all test cases in the regression test to obtain a test result file with a timestamp when the regression test is detected to be over.

[0039] The parsing module is used to parse the regression test results corresponding to all the test cases in the test result file to determine the failed test cases that failed the regression test and the failure reasons corresponding to the failed test cases;

[0040] The classification module is used to classify the failure reasons corresponding to the failed test cases to determine the failure type corresponding to the failed test cases;

[0041] The filtering module is used to filter the failed test cases in the test result file according to the failure type and a preset filtering rule to obtain a filtered test result file.

[0042] The second packaging module is used to repackage the filtered test result file to obtain a filtered test result file carrying a new timestamp.

[0043] Thirdly, the present invention discloses an electronic device, comprising:

[0044] Memory, used to store computer programs;

[0045] A processor is used to execute the computer program to implement the steps of the aforementioned disclosed regression test control method.

[0046] Fourthly, the present invention discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the steps of the aforementioned disclosed regression test control method.

[0047] As can be seen, the present invention provides a regression test control method applied to a regression test tool, comprising: receiving an adjusted configuration file imported by a user; the adjusted configuration file being a configuration file obtained by adjusting the original configuration file according to the current test requirements; refreshing the local old configuration file using the adjusted configuration file, and executing regression tests corresponding to the current test requirements according to the refreshed configuration file; upon detecting the end of the regression test, packaging the regression test results corresponding to all test cases in the regression test to obtain a test result file carrying a timestamp; parsing the regression test results corresponding to all test cases in the test result file to determine the failed test cases that failed the regression test and the failure reasons corresponding to the failed test cases, and classifying the failure reasons corresponding to the failed test cases to determine the failure type corresponding to the failed test cases; filtering the failed test cases in the test result file according to the failure type and a preset filtering rule to obtain a filtered test result file, and repackaging the filtered test result file to obtain a filtered test result file carrying a new timestamp. Therefore, this invention dynamically refreshes the old configuration file using the user-imported adjusted configuration file, and then executes regression tests based on the refreshed configuration file, thereby achieving dynamic control of test case execution. This effectively reduces manual intervention. After the test, all test results corresponding to the test cases are packaged with timestamps to obtain test result files with timestamps, thus preventing test case data from being overwritten. Then, by analyzing the test result files with timestamps, the failure reasons of regression failure test cases are classified and filtered, realizing automatic classification of the failure reasons of failed test cases, further improving verification efficiency. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0049] Figure 1 This is a flowchart of a regression test control method disclosed in this invention;

[0050] Figure 2 This is a schematic diagram of a configuration file format disclosed in this invention;

[0051] Figure 3 This is a flowchart of a specific regression test control method disclosed in this invention;

[0052] Figure 4 This is a schematic diagram of a specific test case regression control disclosed in this invention;

[0053] Figure 5 This is a schematic diagram illustrating the working principle of a regression result generation module disclosed in this invention;

[0054] Figure 6 This is a schematic diagram illustrating the working principle of a regression merging module disclosed in this invention;

[0055] Figure 7 This is a schematic diagram illustrating the working principle of a regression result analysis module disclosed in this invention;

[0056] Figure 8 This is a schematic diagram of a regression test control device disclosed in this invention;

[0057] Figure 9 This is a structural diagram of an electronic device disclosed in this invention. Detailed Implementation

[0058] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0059] Currently, regression testing tools require manual intervention during execution. Verification personnel must dynamically observe and manually select and add test cases, and there's a lack of analysis of log files during test case execution. This makes it difficult to effectively assist verification personnel in quickly locating the causes of failed test cases, and there's a lack of categorization of these test cases based on their failure reasons. Therefore, this invention provides a regression testing control scheme that effectively reduces manual intervention and automatically categorizes the failure reasons of failed test cases, thereby improving verification efficiency.

[0060] This invention discloses a regression test control method, see [link to relevant documentation]. Figure 1 As shown, this method, applied to regression testing tools, includes:

[0061] Step S11: Receive the adjusted configuration file imported by the user; the adjusted configuration file is a configuration file obtained by adjusting the original configuration file according to the current test requirements.

[0062] In this embodiment, the system receives an adjusted configuration file imported by the user, for example, through a pre-built local file import interface. It is understood that the testing tool reserves an input interface for the configuration file so that verification personnel can import the corresponding adjusted configuration file according to testing needs, thereby dynamically controlling the test cases that require regression testing. In other words, users import custom configuration files through the interface reserved by the regression tool, allowing verification personnel to modify the configuration file at any time according to their needs, achieving dynamic configuration of the test cases participating in regression testing. Configurable parts may include the names of the test cases that need to participate in regression testing and the number of random seeds for the test cases, etc.

[0063] It should be noted that the adjusted configuration file is obtained by adjusting the original configuration file according to the current testing requirements, and the original configuration file is generated based on the basic information extracted from the original test plan. The basic information includes the verification target group to which the test case belongs, the test case name, the number of randomization attempts, and the test case execution flag. For test cases that need to participate in regression testing, basic information is extracted from the existing original test plan, and the original configuration file is generated based on this basic information. That is, the basic information used to generate the original configuration file is extracted from the original test plan. The basic information to be extracted includes the verification target group (VO), the test case name (tcName), the number of randomization attempts (randNum), and the test case execution flag (isExecute). The format of the original configuration file is as follows: Figure 2As shown, the verification personnel can modify the random number of tests and test case execution flags in the configuration file according to their needs, avoiding frequent modifications to the original test plan file, achieving dynamic control over the current regression, and enabling control over the currently executed test cases and test cases to be executed both before and after the test case regression test begins.

[0064] Step S12: Refresh the local old configuration file using the adjusted configuration file, and execute regression tests corresponding to the current test requirements based on the refreshed configuration file.

[0065] In this embodiment, after receiving the adjusted configuration file imported by the user, the regression testing tool will refresh the configuration file according to the current testing requirements. That is, it will refresh the local old configuration file using the adjusted configuration file to dynamically control the test cases to be executed. Especially for test cases that need to participate in multiple randomizations, this can be achieved by adjusting the random seed number in the original configuration file. This avoids the need for verifiers to manually click and select the test cases to be regressed at regular intervals, effectively reducing the need for frequent manual control of test case execution, thereby improving the accuracy of the verification work and accelerating the convergence of coverage.

[0066] Step S13: When the regression test is detected to be over, the regression test results corresponding to all test cases in the regression test are packaged to obtain a test result file with a timestamp.

[0067] In this embodiment, when the current regression test ends, the regression test results corresponding to all test cases in the regression test are packaged by a local preset monitor to obtain a test result file with a timestamp. In other words, the regression test results corresponding to all test cases participating in the current regression test are packaged into a file path marked by a timestamp. The test case regression path marked by a timestamp can avoid test data loss and avoid test case logs being overwritten, and can also make it easier for users to view.

[0068] Furthermore, after the backtest is detected to have ended, the process can also include: acquiring historical simulation test data corresponding to the historical regression tests, and acquiring the latest simulation test data corresponding to the current regression test; merging the historical simulation test data and the latest simulation test data to obtain merged simulation test data, and packaging the merged simulation test data to obtain a simulation test data file carrying a timestamp. Specifically, when the locally preset monitor detects that the current regression test has ended, it opens the locally preset first data acquisition interface and second data acquisition interface; it acquires the historical simulation test data corresponding to the historical regression tests through the first data acquisition interface, and acquires the latest simulation test data corresponding to the current regression test through the second data acquisition interface. It is understood that the simulation test data file can be provided to verification personnel for analysis to improve coverage, and timely merging of simulation test data can prevent simulation data from being overwritten or lost after the environment is recompiled.

[0069] Step S14: Parse the regression test results corresponding to all test cases in the test result file to determine the failed test cases that failed the regression test and the failure reasons corresponding to the failed test cases, and classify the failure reasons corresponding to the failed test cases to determine the failure type corresponding to the failed test cases.

[0070] In this embodiment, when the testing tool detects that all test cases have completed regression testing, it simultaneously parses the regression test results corresponding to all test cases in the test result file marked with timestamps to determine the failed test cases that failed the regression test and the failure reasons corresponding to the failed test cases. Then, it classifies the failure reasons corresponding to the failed test cases to determine the failure type corresponding to the failed test cases. Understandably, in the current regression testing, after all test cases have been executed, the execution logs of the test cases are retrieved and analyzed based on the execution results to identify the failed test cases and their reasons for failure. This allows for the automatic categorization of failure issues in regression testing, enabling rapid problem classification of failed test cases and avoiding the need for individual problem localization, thus improving verification efficiency and facilitating the iteration of the test project. Furthermore, during the classification of the failure reasons corresponding to the failed test cases, the first failed test case that differs from other failure types is identified as the header test case of that failure type, facilitating subsequent filtering operations. Additionally, error and timeout feedback mechanisms can be used to categorize and sort the problem reasons based on the test results to form a list. For test cases that time out, timely feedback is provided to remind verification personnel, preventing related test cases from occupying simulation resources for extended periods.

[0071] Step S15: Based on the failure type, filter the failed test cases in the test result file according to the preset filtering rules to obtain a filtered test result file, and repackage the filtered test result file to obtain a filtered test result file carrying a new timestamp.

[0072] In this embodiment, after automatically classifying the failure reasons of failed test cases, the failed test cases in the test result file are filtered according to preset filtering rules based on the failure type to obtain a filtered test result file. This filters failed test cases with the same failure reason and then repackages the filtered test result file to obtain a filtered test result file with a new timestamp. It can be understood that the regression testing tool traverses the file path marked by timestamps, analyzes the test case logs under that file path, classifies and filters the failed test cases, facilitating further debugging by verification personnel, improving verification efficiency, and reducing the server space occupied by test case logs and other files.

[0073] In this embodiment, filtering the failed test cases in the test result file according to the failure type and a preset filtering rule to obtain a filtered test result file may specifically include: sequentially determining whether the failed test cases and the target header test cases in the test result file belong to the same verification target group; the target header test case is a header test case with the same failure type as the failed test cases; if the failed test cases and the target header test cases do not belong to the same verification target group, then the failed test cases are retained; if the failed test cases and the target header test cases belong to the same verification target group, then determining whether the failed test cases belong to test cases that are randomly executed multiple times in regression testing; if the failed test cases do not belong to test cases that are randomly executed multiple times in regression testing, then the failed test cases are retained. If the failed test case is a test case that is randomly executed multiple times in regression testing, then it is determined whether the failure reason keyword of the failed test case is consistent with the failure reason keyword of the target header test case; if the failure reason keyword of the failed test case is consistent with the failure reason keyword of the target header test case, then the failed test case is filtered; if the failure reason keyword of the failed test case is inconsistent with the failure reason keyword of the target header test case, then the failed test case is retained.

[0074] As can be seen, in this embodiment of the invention, the old configuration file is dynamically refreshed using the user-imported adjusted configuration file, and then regression testing is performed according to the refreshed configuration file, thereby achieving dynamic control of test case execution. This effectively reduces manual intervention. After the test is completed, the test results corresponding to all test cases are packaged with timestamps to obtain test result files with timestamps, thus preventing test case data from being overwritten. Then, by analyzing the test result files with timestamps, the failure reasons of regression failure test cases are classified and filtered, realizing automatic classification of the failure reasons of failed test cases, further improving verification efficiency.

[0075] See Figure 3 As shown, this embodiment of the invention discloses a specific regression test control method. Compared with the previous embodiment, this embodiment further explains and optimizes the technical solution.

[0076] Step S21: Receive the adjusted configuration file imported by the user; the adjusted configuration file is a configuration file obtained by adjusting the original configuration file according to the current test requirements.

[0077] Step S22: Refresh the local old configuration file using the adjusted configuration file, extract the target basic information corresponding to each test case from the refreshed configuration file, and store the target basic information in the form of key-value pairs into the pre-built target dictionary.

[0078] It is understandable that the system receives the adjusted configuration file imported by the user, refreshes the local old configuration file using the adjusted configuration file, reads the refreshed configuration file to extract the target basic information corresponding to each test case, and stores the target basic information in a dictionary in the form of key-value pairs.

[0079] Step S23: Traverse the target dictionary to store the random number of each test case required in the current regression test and the test case execution flag in the pre-built test case table.

[0080] In this embodiment, the target dictionary is traversed, and the random number of times and test case execution flags corresponding to each test case required in the current regression test are stored in a list.

[0081] Step S24: Execute regression tests corresponding to the current test requirements according to the test case table.

[0082] In this embodiment, regression tests corresponding to the current test requirements are executed according to the test case table, thereby achieving dynamic control of the test cases to be executed and effectively reducing the need for frequent manual control of test case execution.

[0083] Step S25: When the regression test is detected to be over, the regression test results corresponding to all test cases in the regression test are packaged to obtain a test result file with a timestamp.

[0084] Step S26: Parse the regression test results corresponding to all test cases in the test result file to determine the failed test cases that failed the regression test and the failure reasons corresponding to the failed test cases, and classify the failure reasons corresponding to the failed test cases to determine the failure type corresponding to the failed test cases.

[0085] Step S27: Based on the failure type, filter the failed test cases in the test result file according to the preset filtering rules to obtain a filtered test result file, and repackage the filtered test result file to obtain a filtered test result file carrying a new timestamp.

[0086] For details regarding steps S21, S25, and S27, please refer to the corresponding content disclosed in the foregoing embodiments, which will not be repeated here.

[0087] As can be seen, in this embodiment of the invention, the old configuration file is dynamically refreshed using the user-imported adjusted configuration file, and then regression testing is performed according to the refreshed configuration file, thereby achieving dynamic control of test case execution. This effectively reduces manual intervention. After the test is completed, the test results corresponding to all test cases are packaged with timestamps to obtain test result files with timestamps, thus preventing test case data from being overwritten. Then, by analyzing the test result files with timestamps, the failure reasons of regression failure test cases are classified and filtered, realizing automatic classification of the failure reasons of failed test cases, further improving verification efficiency.

[0088] For a specific application scenario, this invention also discloses a specific test case regression testing method, see [link to relevant documentation]. Figure 4As shown, after extracting basic information from the original test plan, the verification personnel modify the configuration file as needed and import the adjusted configuration file into the regression testing tool. The testing tool includes a regression result generation module (resultGen), a regression result merging module (resultMrg), and a regression result parsing module (resultAnal). The regression result generation module is responsible for packaging the regression test results generated in each round of regression with timestamps. The regression result merging module is responsible for merging the latest simulation test data with historical simulation test data to generate a .vdb file. The regression result parsing module parses the latest generated regression test results, classifies and filters the failed regression test cases, generates a set of failed test cases, and packages them with corresponding timestamps, thereby realizing a lightweight regression testing environment and ultimately achieving closed-loop management of regression test cases. In other words, it realizes the analysis of regression test results and presents the classified and filtered results in a lightweight manner, which can greatly reduce the workload of verification personnel to check and locate the cause of test case failures one by one, improve debugging efficiency, shorten the verification cycle, and at the same time greatly reduce the server resource and space occupation caused by test case logs.

[0089] The working principle of the regression result generation module is described in [reference needed]. Figure 5 As shown, the regression result generation module internally includes a configuration file analysis unit (cfgAnal), an instruction sending unit (cmdSend), and a timestamp log generation unit (newLogGen). For the configuration file analysis unit, after the configuration file is input, it reads the configuration file to extract the target basic information corresponding to each test case and stores the target basic information in a dictionary in key-value pairs. For the instruction sending unit, when a configuration file refresh is detected, it traverses the dictionary and stores the required number of random attempts (randNum) and the test case execution flag (isExecute) in a list. For the timestamp log generation unit, after the current regression test is completed, the regression testing tool sends the completion flag (run_end) to this unit. After detecting that the flag is valid, this unit packages the regression test results generated by the current regression test with a timestamp as the identifier, that is, renames the original regression test result file directory with the timestamp as the identifier, and archives the file directory carrying the timestamp.

[0090] The working principle of the regression result merging module is described in [reference needed]. Figure 6As shown, when the regression testing tool's monitor (refreshMon) detects a configuration file refresh and waits for the current regression test to finish, it opens the interfaces oriVdbIf and newVdbIf. The oriVdbIf interface receives the historical .vdb database, and the newVdbIf interface receives the new .vdb database obtained from the current regression test. When data is input to these two interfaces, a merge operation is performed to obtain a merged .vdb database. The current time is then obtained, and the merged .vdb database is packaged to obtain a merged .vdb database with a timestamp. This results in a newly generated .vdb file with a timestamp, which can be provided to verification personnel for analysis to improve coverage. Timely merging operations prevent related files from being overwritten or lost after environment recompilation. In other words, automatically merging new and old data avoids frequent manual merging by verification personnel, quickly converging coverage while also preventing .vdb data loss due to human error.

[0091] The working principle of the regression result analysis module can be found in [link to documentation]. Figure 7As shown, the regression result parsing module includes a failure reason parsing unit (reasonAnal) and a test case filtering unit (filterCase). The regression result generation module inputs the entire test result file with timestamps into the regression result parsing module. The regression result parsing module first reads the test result file, traverses the test results of all test cases in the test result file to extract and number the verification target group and test case name, and extracts the test case seed number and failure reason keywords, etc., and stores the extracted information in oriListFailed. Next, the failure reason parsing unit (reasonAnal) will parse the failed test cases that failed the regression test and the corresponding failure reasons of the failed test cases according to oriListFailed, and classify them. The first test case of a different category is recorded as the head test case of each category. For other failed regression test cases, the test case filtering unit (filterCase) will filter them according to their category. Each failed test case is compared with the header test case of the same category to determine if the failed test case and the target header test case belong to the same verification target group. If they belong to the same verification target group, it is determined whether the failed test case is a test case that was randomly executed multiple times in the regression test. If it is, the failure reason keyword of the failed test case is compared with the failure reason keyword of the target header test case. If they match, it is filtered out; otherwise, it is retained. This process continues until all failed test cases have been parsed. Finally, the test case filtering unit (filterCase) stores the filtered and retained file information, packaging the corresponding test case results with timestamps.

[0092] Accordingly, embodiments of the present invention also disclose a regression test control device, see [link to relevant documentation]. Figure 8 As shown, the device, used in regression testing tools, includes:

[0093] The receiving module 11 is used to receive the adjusted configuration file imported by the user; the adjusted configuration file is a configuration file obtained by adjusting the original configuration file according to the current test requirements;

[0094] The refresh module 12 is used to refresh the local old configuration file using the adjusted configuration file;

[0095] Test execution module 13 is used to execute regression tests corresponding to the current test requirements based on the refreshed configuration file;

[0096] The first packaging module 14 is used to package the regression test results corresponding to all test cases in the regression test to obtain a test result file with a timestamp when the regression test is detected to be over.

[0097] The parsing module 15 is used to parse the regression test results corresponding to all the test cases in the test result file to determine the failed test cases that failed the regression test and the failure reasons corresponding to the failed test cases;

[0098] The classification module 16 is used to classify the failure reasons corresponding to the failed test cases to determine the failure type corresponding to the failed test cases;

[0099] Filtering module 17 is used to filter the failed test cases in the test result file according to the failure type and a preset filtering rule to obtain a filtered test result file;

[0100] The second packaging module 18 is used to repackage the filtered test result file to obtain a filtered test result file carrying a new timestamp.

[0101] As can be seen from the above, in this embodiment of the invention, the old configuration file is dynamically refreshed by the user-imported adjusted configuration file, and then regression testing is performed according to the refreshed configuration file, thereby realizing dynamic control of test case execution. This can effectively reduce manual intervention. After the test is completed, the test results corresponding to all test cases are packaged with timestamps to obtain test result files with timestamps, thereby preventing test case data from being overwritten. Then, by analyzing the test result files with timestamps, the failure reasons of regression failure test cases are classified and filtered, realizing automatic classification of the failure reasons of failed test cases, further improving verification efficiency.

[0102] In some specific embodiments, the test execution module 13 may specifically include:

[0103] The information extraction submodule is used to extract the target basic information corresponding to each test case from the refreshed configuration file, and store the target basic information in the form of key-value pairs into a pre-built target dictionary;

[0104] The traversal submodule is used to traverse the target dictionary to store the random number of each test case required in the current regression test and the test case execution flag in the pre-built test case table;

[0105] The test execution submodule is used to execute regression tests corresponding to the current test requirements based on the test case table.

[0106] In some specific embodiments, the classification module 16 may specifically include:

[0107] The Header Test Case Determination Submodule is used to determine the first failure test case that appears and is different from other failure types as the header test case of the failure type corresponding to the failure test case.

[0108] In some specific embodiments, the filtering module 17 may specifically include:

[0109] The first judgment submodule is used to sequentially determine whether the failed test cases and the target header test cases in the test result file belong to the same verification target group; the target header test case is a header test case with the same failure type as the failed test cases;

[0110] The second judgment submodule is used to determine whether the failed test case belongs to the test case that is randomly executed multiple times in regression testing when the failed test case and the target header test case belong to the same verification target group.

[0111] The third judgment submodule is used to determine whether the failure reason keyword of the failed test case is consistent with the failure reason keyword of the target header test case when the failed test case belongs to the test cases that are randomly executed multiple times in regression testing.

[0112] The filtering submodule is used to filter the failed test case when the failure reason keyword of the failed test case is consistent with the failure reason keyword of the target header test case.

[0113] The first retention submodule is used to retain the failed test case when the failure reason keyword of the failed test case is inconsistent with the failure reason keyword of the target header test case.

[0114] In some specific embodiments, the filtering module 17 may specifically include:

[0115] The second retention submodule is used to retain the failed test case if the failed test case and the target header test case do not belong to the same verification target group.

[0116] In some specific embodiments, the filtering module 17 may specifically include:

[0117] The second retention submodule is used to retain the failed test case if it does not belong to the test cases that are randomly executed multiple times in regression testing.

[0118] In some specific embodiments, the regression test control device may further include:

[0119] The data acquisition module is used to acquire historical simulation test data corresponding to historical regression tests and the latest simulation test data corresponding to the current regression test.

[0120] The data merging module is used to merge the historical simulation test data and the latest simulation test data to obtain merged simulation test data, and to package the merged simulation test data to obtain a simulation test data file carrying a timestamp.

[0121] In some specific embodiments, the data acquisition module may specifically include:

[0122] The interface enabling submodule is used to enable the first and second data acquisition interfaces locally when the local preset monitor detects that the current regression test has ended.

[0123] The first data acquisition submodule is used to acquire historical simulation test data corresponding to historical regression test through the first data acquisition interface;

[0124] The second data acquisition submodule is used to acquire the latest simulation test data corresponding to the current regression test through the second data acquisition interface.

[0125] Furthermore, embodiments of the present invention also provide an electronic device. Figure 9 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of the invention.

[0126] Figure 9 This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of the present invention. Specifically, the electronic device 20 may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the regression test control method disclosed in any of the foregoing embodiments. Alternatively, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0127] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this invention, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0128] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0129] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the regression test control method executed by the electronic device 20 as disclosed in any of the foregoing embodiments, the computer program 222 may further include a computer program capable of performing other specific tasks.

[0130] Furthermore, this embodiment of the invention also discloses a computer-readable storage medium storing a computer program, which, when loaded and executed by a processor, implements the regression test control method steps disclosed in any of the foregoing embodiments.

[0131] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0132] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0133] The foregoing has provided a detailed description of the regression test control method, apparatus, device, and storage medium provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A regression test control method, characterized in that, Applications to regression testing tools include: Receive the modified configuration file imported by the user; the modified configuration file is a configuration file obtained by adjusting the original configuration file according to the current test requirements; The local old configuration file is refreshed using the adjusted configuration file, and regression tests corresponding to the current test requirements are executed based on the refreshed configuration file. Upon detecting the end of the regression test, the regression test results corresponding to all test cases in the regression test are packaged to obtain a test result file with a timestamp. The regression test results corresponding to all test cases in the test result file are parsed to determine the failed test cases that failed the regression test and the failure reasons corresponding to the failed test cases. The failure reasons corresponding to the failed test cases are then classified to determine the failure type corresponding to the failed test cases. Based on the failure type, the failed test cases in the test result file are filtered according to the preset filtering rules to obtain the filtered test result file, and the filtered test result file is repackaged to obtain the filtered test result file with a new timestamp. The process of classifying the failure reasons corresponding to the failed test cases also includes: The first failure test case that appears that is different from other failure types is identified as the header test case of the failure type corresponding to the failure test case. Accordingly, the step of filtering the failed test cases in the test result file according to the failure type and a preset filtering rule to obtain a filtered test result file includes: The test results file is sequentially checked to determine whether the failed test cases and the target header test cases belong to the same verification target group; the target header test case is a header test case with the same failure type as the failed test cases. If the failed test case and the target header test case belong to the same verification target group, then determine whether the failed test case belongs to the test cases that are randomly executed multiple times in regression testing; If the failed test case is a test case that is randomly executed multiple times in regression testing, then determine whether the failure reason keyword of the failed test case is consistent with the failure reason keyword of the target header test case. If the failure reason keyword of the failed test case is consistent with the failure reason keyword of the target header test case, then the failed test case is filtered. If the failure reason keyword of the failed test case is inconsistent with the failure reason keyword of the target header test case, then the failed test case is retained.

2. The regression test control method according to claim 1, characterized in that, The original configuration file is generated based on the basic information extracted from the original test plan; the basic information includes the verification target group to which the test case belongs, the test case name, the number of randomizations, and the test case execution flag.

3. The regression test control method according to claim 1, characterized in that, The step of executing regression tests corresponding to the current test requirements based on the refreshed configuration file includes: Extract the basic target information corresponding to each test case from the refreshed configuration file, and store the basic target information in the form of key-value pairs into the pre-built target dictionary; The target dictionary is traversed to store the random number of each test case required in the current regression test and the test case execution flag in the pre-built test case table; Execute regression tests corresponding to the current test requirements based on the test case table.

4. The regression test control method according to claim 1, characterized in that, After sequentially determining whether the failed test cases and target header test cases in the test result file belong to the same verification target group, the method further includes: If the failed test case does not belong to the same verification target group as the target header test case, then the failed test case is retained; Accordingly, after determining whether the failed test case belongs to the category of test cases that are randomly executed multiple times in regression testing, the method further includes: If the failed test case is not one of the test cases that are randomly executed multiple times in regression testing, then the failed test case is retained.

5. The regression test control method according to any one of claims 1 to 4, characterized in that, Also includes: Obtain historical simulation test data corresponding to the historical regression test, and obtain the latest simulation test data corresponding to the current regression test; The historical simulation test data and the latest simulation test data are merged to obtain merged simulation test data, and the merged simulation test data is packaged to obtain a simulation test data file with a timestamp.

6. The regression test control method according to claim 5, characterized in that, The step of obtaining historical simulation test data corresponding to historical regression tests and obtaining the latest simulation test data corresponding to the current regression test includes: When the local preset monitor detects that the current regression test has ended, it will open the local preset first data acquisition interface and second data acquisition interface. The first data acquisition interface is used to obtain historical simulation test data corresponding to the historical regression test, and the second data acquisition interface is used to obtain the latest simulation test data corresponding to the current regression test.

7. A regression test control device, characterized in that, Applications to regression testing tools include: The receiving module is used to receive the adjusted configuration file imported by the user; the adjusted configuration file is a configuration file obtained by adjusting the original configuration file according to the current test requirements; A refresh module is used to refresh the local old configuration file using the adjusted configuration file; The test execution module is used to execute regression tests corresponding to the current test requirements based on the refreshed configuration file. The first packaging module is used to package the regression test results corresponding to all test cases in the regression test to obtain a test result file with a timestamp when the regression test is detected to be over. The parsing module is used to parse the regression test results corresponding to all the test cases in the test result file to determine the failed test cases that failed the regression test and the failure reasons corresponding to the failed test cases; The classification module is used to classify the failure reasons corresponding to the failed test cases to determine the failure type corresponding to the failed test cases; The filtering module is used to filter the failed test cases in the test result file according to the failure type and a preset filtering rule to obtain a filtered test result file. The second packaging module is used to repackage the filtered test result file to obtain a filtered test result file carrying a new timestamp. The process of classifying the failure reasons corresponding to the failed test cases also includes: The first failure test case that appears that is different from other failure types is identified as the header test case of the failure type corresponding to the failure test case. Accordingly, the step of filtering the failed test cases in the test result file according to the failure type and a preset filtering rule to obtain a filtered test result file includes: The test results file is sequentially checked to determine whether the failed test cases and the target header test cases belong to the same verification target group; the target header test case is a header test case with the same failure type as the failed test cases. If the failed test case and the target header test case belong to the same verification target group, then determine whether the failed test case belongs to the test cases that are randomly executed multiple times in regression testing; If the failed test case is a test case that is randomly executed multiple times in regression testing, then determine whether the failure reason keyword of the failed test case is consistent with the failure reason keyword of the target header test case. If the failure reason keyword of the failed test case is consistent with the failure reason keyword of the target header test case, then the failed test case is filtered. If the failure reason keyword of the failed test case is inconsistent with the failure reason keyword of the target header test case, then the failed test case is retained.

8. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the regression test control method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, Used to store a computer program; wherein, when the computer program is executed by a processor, it implements the steps of the regression test control method as described in any one of claims 1 to 6.

Images