Regression testing method, system, device and computer storage medium

Abstract

An embodiment of the present invention provides a regression testing method, including: generating regression test cases according to scenario logs; setting a trigger for regression testing, and when the trigger is triggered, driving a business system to execute business logic of a specific scenario and recording regression logs; comparing the regression test cases with the regression logs and recording the comparison results. The embodiment of the present invention neither requires testers to create a large number of regression test cases nor requires too much intervention from testers, thus achieving the purpose of saving manpower. The embodiment of the present invention also provides a regression testing system, device, and computer storage medium.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular, to a testing method, a testing device, and a computer storage medium. Background Art

[0002] Following the rapid development of the Internet, the number of application systems is increasing day by day. As a result, interactions between various systems are required to complete the entire operation, and the relationship is transformed into interactions between sub-microservices that are independent of external business systems.

[0003] With the increase in the number of dependent interfaces, an important issue in the development of each function iteration is to ensure the correctness of the original process. There are the following two ideas:

[0004] Method 1: Testers conduct a large number of regression tests to verify that the small versions launched after iteration do not affect the results of the old process. The disadvantage of this method is that as the number of interfaces provided by external microservices and the interface iteration frequency increase, the regression workload becomes larger and larger. Most of the work is repetitive. Moreover, if there are multiple branches for parallel testing, regression testing needs to be performed on each branch, and it cannot be guaranteed whether the test scenarios cover all the interfaces of external microservices called by the old process. This method requires a large amount of manpower, and the input is often greater than the output.

[0005] Method 2: Adopt the mock testing method, that is, construct test scenarios and input and output parameter data to verify whether all external interfaces can achieve the expected return. The disadvantage of this method is that it cannot guarantee a relatively high coverage rate.

[0006] In summary, the industry needs a better regression testing method. Summary of the Invention

[0007] In view of this, an embodiment of the present invention provides a regression testing method, which generates regression test cases according to stable scenario logs and triggers regression testing through a trigger to solve the problems in the prior art.

[0008] According to the first aspect of the embodiments of the present invention, a regression testing method is provided, including:

[0009] Generating regression test cases according to scenario logs;

[0010] Setting a trigger for regression testing. When the trigger is triggered, driving the business system to execute the business logic of a specific scenario and recording regression logs;

[0011] Comparing the regression test cases with the regression logs and recording the comparison results.

[0012] Optionally, the regression test case for the specific scenario starts from an entry interface. The driving business system to execute the business logic of the specific scenario includes: obtaining the input parameter values of the entry interface from the regression test case, and calling the entry interface based on the input parameter values.

[0013] Optionally, the business logic of the scenario includes calls to multiple interfaces. The recording of the regression log includes: recording the input parameter values of the multiple interfaces.

[0014] Then, the comparison between the regression test case and the regression log includes: comparing the input parameter values of the multiple interfaces recorded in the regression log with the input parameter values of the corresponding interfaces in the regression test case.

[0015] Optionally, the multiple interfaces include local interfaces and external interfaces. The multiple external interfaces are provided with playback buttons. When the playback buttons are activated, the corresponding external interfaces are not called, and the output parameter values of the external interfaces in the regression test case are used to drive the business logic.

[0016] Optionally, a log identifier is used in the regression test case to identify an actual call process in a specific scenario.

[0017] Optionally, the generation of the regression test case according to the scenario log includes:

[0018] Obtaining the partial scenario log that meets the requirements from the scenario log;

[0019] Storing the interface information extracted from the partial scenario log into a database table.

[0020] Optionally, it further includes: completing the information required for the regression test case.

[0021] According to the second aspect of the embodiments of the present invention, a regression test system is provided, including:

[0022] A use case production module, configured to generate a regression test case according to a scenario log;

[0023] A trigger module, configured to set a trigger for the regression test. When the trigger is triggered, it drives the business system to execute the business logic of a specific scenario and records a regression log;

[0024] A comparison module, configured to compare the regression test case and the regression log and record the comparison result.

[0025] Optionally, the regression test for the specific scenario starts from an entry interface. The trigger module includes: obtaining the input parameter values of the entry interface from the regression test case, and calling the entry interface based on the input parameter values.

[0026] Optionally, the business logic of the specific scenario includes calls to multiple interfaces, and the recording of regression logs includes: recording the input parameter values of the multiple interfaces.

[0027] The comparison module includes: comparing the input parameter values of the multiple interfaces recorded in the regression log with the input parameter values of the corresponding interfaces in the regression test case.

[0028] Optionally, the multiple interfaces include local interfaces and external interfaces, and the regression test system further includes: a playback module, which sets a playback button for each external interface. When the playback button is activated, the corresponding external interface is not called, and the output parameter values of the external interfaces in the regression test case are used to drive the business logic.

[0029] According to a third aspect of an embodiment of the present invention, there is provided a regression test device, including:

[0030] A memory for storing computer instructions;

[0031] A processor coupled to the memory, and the processor is configured to execute to implement the above regression test method based on the computer instructions stored in the memory.

[0032] According to a fourth aspect of an embodiment of the present invention, there is provided a computer-readable storage medium storing computer instructions, and when the computer instructions are executed, the above regression test method is implemented.

[0033] The embodiment of the present invention has the following advantages or beneficial effects: A second scoring function is constructed based on the original scoring function and sorting factors, and the weight values are optimized according to the basic data set, without manually configuring different correction item parameters for different categories, thereby improving the construction efficiency of the search model. Further, the weight values are optimized by category, improving the optimization efficiency.

[0034] The preferred embodiment of the present invention has the following advantages or beneficial effects: Regression test cases are generated based on stable scenario logs, and regression tests are triggered by triggers. Neither a large number of regression test cases need to be created by testers nor too much intervention from testers is required, thus achieving the purpose of saving manpower. Moreover, since the scenario logs are logs generated in the production environment, a relatively high coverage rate of regression tests can be ensured without increasing manpower.

[0035] The preferred embodiment of the present invention has the following advantages or beneficial effects: A playback button corresponding to the external interface is set. When each playback button is activated, the corresponding external interface is not called, and the output parameter values in the regression test case are used to drive the business logic, thereby separating the business logic of the business system from the business logic of the external interface and facilitating problem location. Description of the Drawings

[0036] By referring to the description of the embodiments of the present invention with reference to the following drawings, the above and other objects, features, and advantages of the present invention will become more apparent. In the drawings:

[0037] Figure 1 is a flowchart of the regression testing method according to an embodiment of the present invention;

[0038] Figure 2 is a flowchart of the business logic of an exemplary business system in the prior art;

[0039] Figure 3 is a flowchart of generating regression test cases for the regression testing method according to an embodiment of the present invention;

[0040] Figure 4 is a schematic diagram of the database table structure for storing regression test cases according to an embodiment of the present invention;

[0041] Figure 5 is a flowchart of the regression testing method according to an embodiment of the present invention;

[0042] Figure 6 is a structural diagram of the regression testing system according to an embodiment of the present invention;

[0043] Figure 7 is a structural diagram of the backtest testing device according to an embodiment of the present invention. Detailed Embodiments

[0044] The following describes the present invention based on embodiments, but the present invention is not limited to these embodiments. In the following detailed description of the present invention, some specific details are described in detail. Those skilled in the art can fully understand the present invention without the description of these details. In order to avoid obscuring the essence of the present invention, well-known methods, processes, and procedures are not described in detail. In addition, the drawings are not necessarily drawn to scale.

[0045] Figure 1 is a flowchart of the regression testing method according to an embodiment of the present invention. When the code of the business system changes, it is necessary to perform testing based on the regression testing method provided in this embodiment. The method includes the following steps.

[0046] In step S101, regression test cases are generated according to the scenario logs.

[0047] In step S102, a trigger for the regression test is set. When the trigger is triggered, it drives the business system to execute the business logic of a specific scenario and records the regression logs.

[0048] In step S103, the regression test cases and the regression logs are compared, and the comparison results are recorded.

[0049] Scenario logs are generated by the business system in the production environment. Scenario logs include logs in multiple scenarios. Logs in the same scenario are multiple logs recorded by the business code triggered by the same request. The business system generally uses a log identifier to identify an actual invocation process in a specific scenario.

[0050] In this embodiment, the scenario logs generated by the business system in the production environment are parsed and processed to obtain multiple regression test cases. When the trigger triggers the regression test, the business system is driven to obtain the regression test cases of a certain scenario from the regression test cases, execute the business logic of the specific scenario based on the regression test cases, record the regression logs, compare the regression logs with the regression test cases, and the comparison result is the regression test result.

[0051] According to the embodiment of the present invention, regression test cases are generated based on stable scenario logs, and the regression test is triggered by a trigger, which neither requires testers to create a large number of regression test cases nor requires too much intervention from testers, thus achieving the purpose of saving manpower. Moreover, since the scenario logs are the logs generated in the production environment, it can ensure a relatively high coverage rate of the regression test without increasing manpower.

[0052] Figure 2 It is a flowchart of the business logic of an exemplary business system in the prior art. In this scenario, the initiating system 201, the business system 202, and the called external interface 203 are involved.

[0053] In step S211, the initiating system 201 initiates a request.

[0054] In step S212, the input parameters of interface 1 are assembled and external interface 1 is called.

[0055] In step S213, the output parameters of interface 1 are received and processed.

[0056] In step S214, other logic is continued

[0057] In step S217, the input parameters of interface 2 are assembled and external interface 2 is called.

[0058] In step S216, the output parameters of interface 2 are received and processed.

[0059] In step S217, after continuing other logic, the parameters are returned.

[0060] In this example, the business system needs to call multiple interfaces in a scenario, and these interfaces include local interfaces and external interfaces. The business system can record the call information of these interfaces in the scenario logs according to a preset specification, so as to facilitate other applications to parse the scenario logs. Of course, the scenario logs can not only record the call information of various interfaces, but also record annotation information, variable value information, etc.

[0061] In one example, a scenario log is recorded in the format of log identifier + space + system signature + comma + parameter type + colon + specific parameter object information. An exemplary scenario log is as follows.

[0062] EWsZY8fLGoQNdIq-Provider#method0, input parameter: XXXXXX

[0063] EWsZY8fLGoQNdIq–Consumer1#method1, input parameter: XXXXXX

[0064] EWsZY8fLGoQNdIq–Consumer1#method1, output parameter: XXXXXX

[0065] EWsZY8fLGoQNdIq–Consumer2#method2, input parameter: XXXXXX

[0066] EWsZY8fLGoQNdIq–Consumer2#method2, output parameter: XXXXXX

[0067] EWsZY8fLGoQNdIq-Provider#method0, output parameter: XXXXXX

[0068] Among them, EWsZY8fLGoQNdIq is the log identifier, Provider#method0 and Consumer1#method1 represent the interface names, Provider represents the name of the business system, which is the system initiating the scenario, and consumer represents the name of the called system.

[0069] Figure 3 is the flowchart for generating regression test cases of the regression test method in an embodiment of the present invention. Specifically, it includes the following steps.

[0070] In step S301, obtain the partial scenario logs that meet the requirements from the scenario logs.

[0071] In step S302, store the interface information extracted from the partial scenario logs into a database table.

[0072] In step S303, complete the information required for the regression test cases.

[0073] In this embodiment, the interface call logs can be extracted from the scenario logs based on the specifications of the interface call logs. Moreover, in order to ensure that the interface call log information of the same scenario is grouped together, regular expressions can be used to match the log identifiers of the interface call logs, and the successfully matched interface call logs can be output. The output interface call logs can be stored in a message queue. These interface call logs can be partially or fully used to generate regression test cases. Optionally, a threshold can be set to determine the proportion of the logs used to generate regression test cases in the original logs. For example, only 80% of the logs in the log queue can be selected, or only the first 10M of the queue can be selected to generate regression test cases, and the rest can be directly cleared and discarded. Optionally, an independent process or thread can be used to report the interface call logs to the message queue, and another independent process or thread can be used to batch parse the interface call logs in the message queue and store them in a database table. When reporting the logs to the message queue or storing the logs in the database table, the information required for the regression test cases should also be completed, such as filling in the application name of the business system, creation time information, etc. In addition, a manual upload method is also provided, allowing users to select log files for upload.

[0074] In one example, there are two database tables for storing regression test cases. One table stores the interface information of multiple scenarios of the business system (i.e., Provider), including the interface name (class name + method name), input parameter values, and output parameter values of the business system. The other table stores the interface information of the called system (i.e., Consumer), including the input parameter values and output parameter values of the external interface. The data structures of these two tables are the same, as Figure 4 shown. Figure 4 is a schematic diagram of an exemplary database table structure for storing regression test cases according to an embodiment of the present invention. Refer to Figure 4 and the table structure includes log identifier, application name, interface name, method name, parameter type (input parameter, output parameter, return parameter, etc.), and parameter value (input parameter value, output parameter value, return value).

[0075] As can be understood from the above description, the regression test cases can only include interface information, and the first interface is the entry interface. According to an embodiment of the present invention, the entry value of the entry interface in the regression test cases is sent to the entry interface in the business system, and the entry interface can automatically execute and generate the output parameter values of the remaining interfaces. By comparing these output parameter values with the output parameter values recorded in the regression test cases, the regression test results can be obtained.

[0076] Figure 5 is a flowchart of the regression test method according to an embodiment of the present invention. Specifically, it includes the following steps.

[0077] In step S500, it is determined whether the playback button is activated. If so, the regression test is triggered, that is, steps S501 - S513 are executed.

[0078] In step S501, the regression test starts and a log identifier is selected. This log identifier corresponds to an actual call process under a specific scenario.

[0079] In step S502, the input parameters of Interface 1 are assembled. That is, the input parameter values of Interface 1 are obtained from the corresponding regression test cases acquired in step S501.

[0080] In step S503, it is determined whether the playback button is activated. If so, step S504 is executed; otherwise, Interface 1 is called based on the input parameter values obtained in step S502.

[0081] In step S504, it is judged whether the input parameters are consistent with the regression test cases. If so, step S505 is executed; otherwise, step S513 is executed.

[0082] In step S505, the output parameters of Interface 1 are processed.

[0083] In step S506, other logic is continued.

[0084] In step S507, the input parameters of Interface 2 are assembled. That is, the input parameter values of Interface 2 are obtained from the corresponding regression test cases acquired in step S501.

[0085] In step S508, it is determined whether the playback button is activated. If so, step S509 is executed; otherwise, Interface 2 is called based on the input parameter values obtained in step S507.

[0086] In step S509, it is judged whether the input parameters are consistent with the regression test cases. If so, step S510 is executed; otherwise, step S513 is executed.

[0087] In step S510, the output parameters of Interface 2 are processed.

[0088] In step S511, it is repeated N times.

[0089] In step S512, the output parameter values are compared.

[0090] In step S513, the verification result is recorded.

[0091] In this embodiment, in order to separate the business logic of the business system from the business logic of the external interfaces, a playback button is specifically set up, including a general playback button and multiple playback buttons corresponding to multiple external interfaces. When the general playback button is turned on, regression testing is started. When the playback button corresponding to each external interface is turned on, the output parameter values of the external interfaces in the regression test cases are used to drive the normal execution of the business logic of the business system. Thus, by controlling the general playback button, it is determined whether to start the regression testing, and by controlling the playback button corresponding to each external interface, it is determined whether to shield the external interfaces in this regression testing. Since the maintainers of the external interfaces and the maintainers of the business system are generally different, by shielding some or all of the external interfaces, it can be determined whether the problems that occur are due to the code problems of the business system itself. Moreover, when the output parameter values of all the external interfaces in the regression testing are replaced with the output parameter values of the corresponding external interfaces in the regression test cases, there will be no actual interaction with the external interfaces, and the external interfaces will not be affected on the basis of simulating the scenario calls at that time (especially for external interfaces with write operation types). In this embodiment, when it is determined that the input parameter values are inconsistent with the corresponding input parameter values in the regression test cases, the verification results are recorded. When the interface calls are repeated N times, the output parameter values are compared, and the verification results are recorded. By reading the verification results, the tester can narrow down the scope of the problematic code in the business system.

[0092] In one embodiment, the corresponding functions of the regression testing can be set in a plug-in, and this plug-in is embedded inside the business system. If the code of the business system is modified, the regression testing can be started in a timely manner.

[0093] In one embodiment, the verification results are displayed. And the business system is scored based on the verification results. If the verification results show that the number of inconsistent input parameter values is large, the score of this regression testing is increased, indicating that the error probability of the business system is relatively high and needs to be focused on in subsequent regression testing, thus providing feedback for subsequent regression testing.

[0094] Figure 6 It is the structural diagram of the regression testing system of the embodiment of the present invention, including a test case production module 601, a trigger module 602, and a comparison module 603.

[0095] The test case production module 601 generates regression test cases according to the stable scenario logs.

[0096] The trigger module 602 sets the trigger for the regression testing. When the trigger is triggered, it drives the business system to execute the business logic of a specific scenario and records the regression logs.

[0097] The comparison module 603 is used to compare the regression test cases and the regression logs and record the comparison results.

[0098] In one embodiment, a specific scenario starts from an entry interface. The trigger module 602 includes: obtaining the input parameter values of the entry interface from the regression test cases, and invoking the entry interface based on the input parameter values.

[0099] In one embodiment, the business logic of the specific scenario includes invocations of multiple interfaces. The recording of regression logs includes: recording the input parameter values of the multiple interfaces.

[0100] The comparison module includes: comparing the input parameter values of the multiple interfaces recorded in the regression logs with the input parameter values of the corresponding interfaces in the regression test cases.

[0101] In one embodiment, the regression test system further includes: a playback module for setting a playback button. When the playback button is activated, the corresponding external interface is not invoked, and the output parameter values of the external interfaces in the regression test cases are used to drive the business logic.

[0102] In one embodiment, the regression test system further includes a regression module for setting a playback button for each of the multiple external interfaces. When the playback button is activated, the corresponding external interface is not invoked, and the output parameter values in the regression test cases are used to drive the business logic.

[0103] Figure 7 It is a structural diagram of a device for constructing a search model according to an embodiment of the present invention. Figure 7 The illustrated device is merely an example and should not impose any limitation on the functions and usage scope of the embodiments of the present invention.

[0104] Reference Figure 7 , the device includes a processor 701, a memory 702, and an input / output device 703 connected via a bus. The memory 702 includes a read-only memory (ROM) and a random access memory (RAM). The memory 702 stores various computer instructions and data required for executing system functions. The processor 701 reads various computer instructions from the memory 702 to perform various appropriate actions and processes. The input / output device includes an input part such as a keyboard and a mouse; an output part including, for example, a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker; a storage part including a hard disk, etc.; and a communication part including a network interface card such as a LAN card and a modem. The memory 702 also stores the following computer instructions to complete the operations specified for the device of the embodiments of the present invention: generating regression test cases according to scenario logs; setting a trigger for regression testing. When the trigger is triggered, the business system obtains the input parameter values of a specific scenario from the regression test cases, drives the business logic based on the input parameter values, and records regression logs; comparing the regression test cases and the regression logs, and recording the comparison results.

[0105] Accordingly, an embodiment of the present invention provides a computer-readable storage medium storing computer instructions, and when the computer instructions are executed, the operations defined by the above method are implemented.

[0106] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions, and operations of the system, method, and apparatus according to the embodiments of the present invention. The boxes in the flowcharts and block diagrams may represent a module, a program segment, or just a piece of code, and the module, program segment, and code are all executable instructions for implementing the defined logical functions. It should also be noted that the executable instructions for implementing the defined logical functions can be recombined to generate new modules and program segments. Therefore, the boxes in the accompanying drawings and the order of the boxes are only used to better illustrate the processes and steps of the embodiments, and should not be construed as a limitation on the invention itself.

[0107] Each module or unit of the system can be implemented by hardware, firmware, or software. Software includes, for example, coding programs formed in various programming languages such as JAVA, C / C++ / C#, SQL, etc. Although the steps and the order of the steps in the embodiments of the present invention are given in the method and the method illustration, the executable instructions for implementing the defined logical functions of the steps can be recombined to generate new steps. The order of the steps should not be limited only to the order of the steps in the method and the method illustration, and can be adjusted at any time according to the needs of the function. For example, some of the steps can be executed in parallel or in the reverse order.

[0108] The system and method according to the present invention can be deployed on a single or multiple servers. For example, different modules can be deployed on different servers to form dedicated servers. Alternatively, the same functional units, modules, or systems can be distributedly deployed on multiple servers to relieve the load pressure. The servers include, but are not limited to, multiple PCs, PC servers, blade servers, supercomputers, etc. in the same local area network and connected through the Internet.

[0109] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A regression testing method, characterized in that, Including: Generating regression test cases based on scenario logs; Setting a trigger for regression testing. When the trigger is triggered, driving the business system to execute the business logic of a specific scenario and recording regression logs; Comparing the regression test cases and the regression logs and recording the comparison results; Wherein, the business logic of the specific scenario includes calls to multiple interfaces, and the recording of the regression logs includes: recording the input parameter values of the multiple interfaces; Then the comparing of the regression test cases and the regression logs includes: comparing the input parameter values of the multiple interfaces recorded in the regression logs with the input parameter values of the corresponding interfaces in the regression test cases; The multiple interfaces include local interfaces and external interfaces. Each external interface is provided with a playback button. When the playback button is activated, the corresponding external interface is not called, and the output parameter values of the external interfaces in the regression test cases are used to drive the business logic, thereby shielding the external interfaces in this regression test.

2. The regression testing method according to claim 1, characterized in that, The regression test cases for the specific scenario start from an entry interface. The driving of the business system to execute the business logic of the specific scenario includes: obtaining the input parameter values of the entry interface from the regression test cases and calling the entry interface based on the input parameter values.

3. The regression testing method according to claim 1, characterized in that, In the regression test cases, a log identifier is used to identify an actual call process in a specific scenario.

4. The regression testing method according to claim 3, characterized in that, The generating of the regression test cases based on the scenario logs includes: Obtaining the partial scenario logs that meet the requirements from the scenario logs; Storing the interface information extracted from the partial scenario logs into a database table.

5. The regression testing method according to claim 1, characterized in that, Also including: Completing the information required for the regression test cases.

6. A regression testing system, characterized in that, Including: A use case production module for generating regression test cases based on scenario logs; A trigger module for setting a trigger for regression testing. When the trigger is triggered, driving the business system to execute the business logic of a specific scenario and recording regression logs; A comparison module for comparing the regression test cases and the regression logs and recording the comparison results; Wherein, the business logic of the specific scenario includes calls to multiple interfaces, and the recording of the regression logs includes: recording the input parameter values of the multiple interfaces; The comparison module includes: comparing the input parameter values of the multiple interfaces recorded in the regression logs with the input parameter values of the corresponding interfaces in the regression test cases; the multiple interfaces include local interfaces and external interfaces, and the regression test system further includes: a playback module for setting a playback button for each external interface. When the playback button is activated, the corresponding external interface is not called, and the output parameter values of the external interfaces in the regression test cases are used to drive the business logic, thereby shielding the external interfaces in this regression test.

7. The regression testing system according to claim 6, characterized in that, The regression test for the specific scenario starts from an entry interface. The trigger module includes: obtaining the input parameter values of the entry interface from the regression test cases and calling the entry interface based on the input parameter values.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions, and when the computer instructions are executed, the regression test method as described in any one of claims 1 to 5 is implemented.

9. A regression testing device, characterized in that, Including: A memory for storing computer instructions; A processor, coupled to the memory, the processor being configured to execute a regression testing method according to any one of claims 1 to 5 based on computer instructions stored in the memory.

Images