Automated Testing Method, System, Device, and Computer-Readable Storage Medium

Abstract

An embodiment of the present invention provides an automated testing method and an automated testing system. The automated testing method includes: generating a plurality of test cases based on interface call logs in a production environment, where each test case includes an interface name, an input parameter value, and a first output parameter value; for each test case, performing an interface call based on the input parameter value and the interface name, obtaining a returned second output parameter value, comparing the first output parameter value with the second output parameter value, and determining whether each test case passes according to the comparison result. The automated testing method does not require testers to design and construct test cases, thereby avoiding the risk of missed testing, and automatically completing the comparison of test results by the system, with high reliability.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and particularly to an automated testing method, system, device, and computer-readable storage medium. Background Art

[0002] Currently, the same set of databases is used in the pre-release environment and the online environment of JD.com. All write data operations are directly stored in the database and will directly enter the production environment. If write interface tests are performed in the pre-release environment and the online environment, the generated test dirty data will enter the production environment. Therefore, generally, interface tests are not allowed in the pre-release environment and the online environment. The current solution is to perform corresponding tests in the test environment. However, the inventor found that there are the following drawbacks in constructing data in the test environment for write interface tests:

[0003] First, the data in the test environment is relatively small and easy to modify. Not only is the maintenance cost high, but it is also difficult to cover completely, and there is a risk of missed testing.

[0004] Second, the test cases are all manually designed, and manual design is inevitably prone to omissions, and there is a risk of missed testing due to incomplete consideration. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide an automated testing method and an automated testing system to solve the problem of missed testing in the prior art.

[0006] According to the first aspect of the present invention, an automated testing method is provided, including:

[0007] Generating a plurality of test cases based on the interface call logs in the production environment, each test case including an interface name, an input parameter value, and a first output parameter value;

[0008] For each test case, performing an interface call based on the input parameter value and the interface name, and obtaining the returned second output parameter value, comparing the first output parameter value with the second output parameter value, and determining whether each test case passes according to the comparison result.

[0009] Preferably, it further includes: storing the plurality of test cases in a test library.

[0010] Preferably, it further includes: maintaining at least one test case through a configuration file.

[0011] Preferably, the performing an interface call for each test case based on the corresponding input parameter value and the interface name includes:

[0012] Sequentially reading each test case from the configuration file to obtain the interface name, reading the corresponding input parameter value from the test library according to the interface name, and then performing the interface call.

[0013] Preferably, the execution interface call includes:

[0014] Set up an interface application, which receives the corresponding entry parameter values and interface names, and calls the corresponding interfaces according to the corresponding entry parameters and interface names.

[0015] Preferably, the interface application is connected to the database in the production environment.

[0016] Preferably, it further includes: dividing the interfaces into read interfaces and write interfaces. For the write interfaces, change the read interfaces they depend on to the official online aliases, and mask the database writing operations in the write interfaces.

[0017] Preferably, the interface name includes the class name and method name.

[0018] Preferably, the generation of multiple test cases includes:

[0019] For each interface call log, determine whether the interface call log contains exit parameter values and / or entry parameter values;

[0020] When an interface call log contains both exit parameter values and entry parameter values, generate a test case based on the interface call log;

[0021] If an interface call log only contains exit parameter values, and if the corresponding record containing entry parameter values is found, generate a test case.

[0022] According to the second aspect of the present invention, an embodiment of the present invention provides an automated testing system, including:

[0023] A use case generation module, configured to generate multiple test cases based on the interface call logs in the production environment. Each test case includes an interface name, entry parameter values, and first exit parameter values;

[0024] An automated testing module, configured to, for each test case, send the entry parameter values and the interface name to the interface application, obtain the returned second exit parameter values, compare the first exit parameter values with the second exit parameter values, and determine whether each test case passes according to the comparison result;

[0025] An interface application, configured to receive the entry parameter values and the interface name from the automated testing module and execute the interface call, and return the second exit parameter values of the interface call to the automated testing module.

[0026] Preferably, the interface application is connected to the database in the production environment.

[0027] Preferably, for the write interface called by the interface application, change the read interface it depends on to the official online alias, and mask the operation of writing to the database in the write interface.

[0028] According to the third aspect of the embodiments of the present invention, there is provided a computer-readable storage medium, characterized in that the computer-readable storage medium stores computer instructions, and when the computer instructions are executed, an automated testing method is implemented.

[0029] According to the fourth aspect of the embodiments of the present invention, there is provided an automated testing system, including:

[0030] A memory for storing computer instructions;

[0031] A processor coupled to the memory, and the processor is configured to execute based on the computer instructions stored in the memory to implement the above-mentioned automated testing method.

[0032] An embodiment of the present invention has the following advantages or beneficial effects: generating test cases required for automation according to the interface call logs in the production environment, and performing automated testing according to the test cases, without the need for testers to design and construct test cases and test data, thus avoiding the risk of missed testing, and automatically comparing the test results by the system, with high reliability.

[0033] A preferred embodiment of the present invention has the following advantages or beneficial effects: connecting the automated testing system to the database in the production environment, and masking the operations in the interface application that will write data to the database, avoiding generating dirty data in the production environment during automated testing while making full use of the real data in the production environment for automated testing. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Through the description of the embodiments of the present invention with reference to the following drawings, the above and other objects, features, and advantages of the present invention will become clearer. In the drawings:

[0035] Figure 1 is a schematic structural diagram of an automated testing system according to an embodiment of the present invention;

[0036] Figure 2 is Figure 1 the specific flowchart of the use case generation module shown;

[0037] Figure 3 is Figure 1 the specific flowchart of the interface transformation module shown;

[0038] Figure 4 is Figure 1 the specific flowchart of the automated testing module 300 shown;

[0039] Figure 5 is a flowchart of an automated testing method according to an embodiment of the present invention;

[0040] Figure 6 is a structural diagram of an automated testing apparatus according to an embodiment of the present invention. Detailed implementation manners

[0041] The present invention will be described based on embodiments hereinafter, but the present invention is not limited to these embodiments only. In the following detailed description of the present invention, some specific details are described in detail. Those skilled in the art can fully understand the present invention without the description of these details. In order to avoid obscuring the essence of the present invention, well-known methods, processes and procedures are not described in detail. Additionally, the drawings are not necessarily drawn to scale.

[0042] Figure 1 is a schematic structural diagram of an interface testing system according to an embodiment of the present invention. The automated testing system 10 includes a test case generation module 100, an interface providing module 200 and an automated testing module 300.

[0043] As Figure 1 shown, the logbook system 103 is a log recording system of an online business system, which is used to record interface call logs and business processing logs in a production environment, and store these logs in a log file or a test library. The logbook system 103 generally records interface call logs according to preset rules, and the test case generation module 100 reads interface call logs from the log file or data table according to the corresponding rules, and generates interface test cases required for automated testing. Not all interface call logs in the production environment can be converted into interface test cases required for automated testing. Interface test cases required for automated testing generally need to provide information such as class name (or interface name), method name, input parameters, output parameters, return parameters, etc. Interface call logs lacking corresponding information will not be converted into test cases required for automated testing. Therefore, only part of the interface call logs in the production environment are converted into interface test cases required for automated testing after being processed.

[0044] The interface application 200 calls multiple interfaces and returns result information. The interface application can be deployed on a specific server, receive the interface name and input parameter values sent by the automated testing module 300, call the corresponding interfaces based on the interface name and input parameter values, and return the output parameter values of the interface calls to the automated testing module 300.

[0045] The automated testing module 300 performs automated testing according to the test cases. The test cases can come from, for example Figure 1The configuration file 401 shown records the interface name of the interface to be tested. According to the interface name, the corresponding test case can be retrieved, the input parameter values of the test case are read, and the input parameter values and the interface name are passed to the interface application, so that the interface application executes the corresponding interface and obtains the return information 302. The return information 302 is the output parameter value of the interface. Then, the automated test module 300 compares the output parameter value obtained by executing the interface with the output parameter 303 in the interface log according to the predetermined comparison rule 402. If they are the same, it means that the test case passes; if they are different, it means that the test case fails. And so on until all the interfaces to be tested in the configuration file 401 are tested. If there is no configuration file 401, the automated test module 300 directly reads the test cases provided by the test case generation module 100 for testing.

[0046] The automated test system provided by the embodiment of the present invention generates the test cases required for automation according to the interface call logs in the production environment, and performs automated tests according to the test cases, without the need for testers to design and construct test cases and test data, thus avoiding the risk of missed testing. The system automatically completes the comparison of test results, with high reliability.

[0047] In one embodiment, the interface application 200 is connected to the database in the production environment. In order to avoid a large amount of dirty data being generated in the production environment when the interface is called, various interfaces called by the interface application are transformed at the code level so that the interface application does not write dirty data into the production environment. Some interfaces will write data into the database, and such interfaces are called write interfaces. Some other interfaces do not involve data writing operations, and such interfaces are called read interfaces. Since read interfaces do not generate dirty data, no code transformation is required. However, code transformation is required for write interfaces. Specifically, the read interfaces on which the write interfaces depend are sorted out, and the aliases of the read interfaces are configured as the aliases set online, and the operations of writing to the database in the write interfaces are shielded (no data is written into the production environment). The interface application after code transformation can be deployed on a specific server, and an alias, such as autotest001, is provided for the transformed interface application.

[0048] It can be seen that connecting the interface application to the database in the production environment and shielding the operations in the interface application that write data into the database can make full use of the real data in the production environment for automated testing and avoid generating dirty data in the production environment during automated testing.

[0049] In one embodiment, the data structure of the test case is shown in the following table.

[0050]

[0051] Among them, uuid represents the identifier of the log, and the uuids of the logs for the same interface call are the same. For example, when the logbook system starts to call an interface, it writes a log containing uuid, class name, method name, and input parameter values. When the interface call is completed, it writes another log containing uuid, class name, method name, output parameter values, and return value. Thus, through the uuid, it can be determined that these two logs are for the same interface. Result represents the output parameter obtained from the interface call log in the production environment, and real_result represents the output parameter value returned based on the interface application. state represents the execution result of the test case. For example, the automated testing module 300 compares result and real_result. If they are consistent, state is set to 1; otherwise, it is set to 0.

[0052] In one embodiment, as Figure 2 shown, Figure 1 the use case generation module shown includes the following steps.

[0053] In step S201, the data in the log file is read one by one.

[0054] In step S202, it is determined whether there is an output parameter value for this record. If there is an output parameter value, step S203 is executed.

[0055] In step S203, it is determined whether there is an input parameter value. If there is an input parameter value, step S204 is executed.

[0056] In step S204, it is directly inserted into the test library.

[0057] In step S205, it is determined whether the uuid of this output parameter record exists in another record in the database. If so, step S207 is executed.

[0058] In step S206, this record is discarded.

[0059] In step S207, the input parameter value and the output parameter value are obtained from the two records and inserted into the test library.

[0060] In step S208, this record is discarded.

[0061] The implementation of this embodiment is based on the data structure in the above table. Specifically, for each record read from the interface call log in the production environment, first, it is determined whether there is an export parameter value. If there is, then it is determined whether there is an import parameter value in this record. If there is also, a test case can be inserted into the test library. If it is determined that this record does not contain an export parameter value, this record is discarded. If this record contains an export parameter value and another record with the same uuid is found in the test library, the import parameter value and the export parameter value are respectively obtained from the two records and inserted into the test library.

[0062] For this embodiment, it should be understood that the processing logic of this embodiment not only depends on the database structure of the above table, but also depends on the log writing rules of the interface call log. If the log writing rules or the database structure change, the processing logic of this embodiment will also change accordingly.

[0063] In one embodiment, as Figure 3 shown, Figure 1 the interface application 200 shown in

[0064] needs to be transformed through the following steps.

[0065] In step S301, sort out the read interfaces on which the write interfaces depend.

[0065] In step S302, change the alias of the read interface to the official online alias.

[0066] In step S303, block all write operations of this interface.

[0067] In step S304, provide a separate alias externally.

[0068] In this embodiment, for the interface application that calls the interface, it is necessary to sort out the read interfaces on which all write interfaces depend, configure the alias of the read interface as the official online alias, block the operation of writing to the database (do not actually write data to the online production environment), and provide a separate alias for the automated test module. To call the interface in the real environment, the alias of the interface must be configured as the official online alias, and the interface-level call is distinguished by the alias. This embodiment realizes the transformation process of the interface application. By blocking the operation of writing to the database, it avoids generating dirty data in the production environment database.

[0069] In one embodiment, as Figure 4 shown, Figure 1 the automated test module 300 shown in

[0070] includes the following steps.

[0071] In step S401, read the interface name and method name in the configuration file.

[0071] In step S402, read the input parameter values and output parameter values of all test cases for the corresponding interface name and method name. The output parameter values are stored in the result field.

[0072] In step S403, execute the interface calls one by one using the input parameter values.

[0073] In step S404, obtain the return result of the call and write it into the real_result field.

[0074] In step S405, compare the value of real_result with the value of the result field according to the comparison rule.

[0075] In step S406, determine whether they are consistent. If they are consistent, execute step S408; otherwise, execute step S407.

[0076] In step S407, this test case fails to execute.

[0077] In step S408, this test case is successfully executed.

[0078] The implementation of this embodiment is based on the data structure in the above table. Specifically, for each record read from the configuration file, according to the interface name of the record, read the input parameter values and output parameter values of the corresponding interface from the test library, store the output parameter values in the corresponding result field, execute the corresponding interface call according to the input parameter values, store the obtained result in the real_result field, compare result and real_result according to the preset rule, thereby determining whether the test case is successfully executed, and store the execution result in the state field.

[0079] Correspondingly, an embodiment of the present invention also provides an automated testing method. As Figure 5 shown, the automated testing method includes the following steps.

[0080] In step S501, based on the interface call logs in the production environment, generate multiple test cases, and each test case includes an interface name, input parameter values, and first output parameter values.

[0081] In step S502, read the test cases.

[0082] In step S503, execute the interface call based on the corresponding input parameter values and interface name, and obtain the returned second output parameter values.

[0083] In step S504, compare the first output parameter values and the second output parameter values, and determine whether each test case passes according to the comparison result.

[0084] In step S505, it is determined whether all test cases have been completed. If all test cases have been completed, the automated test process ends; if not all test cases have been completed, the process returns to step S502 for execution.

[0085] The automated test method provided by the embodiment of the present invention corresponds to the above-mentioned automated test system. By parsing the interface call logs in the production environment to form test cases and then performing automated tests on the test cases, it is not necessary to prepare test cases in advance, reducing the risk of missed tests.

[0086] In a preferred embodiment, the above-mentioned automated test method is directly connected to the database in the production environment. Moreover, in order to avoid generating dirty data in the database of the production environment, some operations in the interface that write data to the database are blocked.

[0087] In a preferred embodiment, a configuration file is set to store test cases. The test cases in the configuration file can be all the test cases obtained based on the interface call logs in the production environment, or can only include some of the test cases generated based on the interface call logs in the production environment.

[0088] In a preferred embodiment, a comparison rule is set to compare the return result of the test case with the return result obtained from the actual interface call logs. The comparison rules are, for example, equality, inclusion, inequality, and expressions.

[0089] In summary, the automated test system and the automated test method provided by the embodiments of the present invention use the interface call logs in the production environment as the input parameters for automated testing, can cover a large amount of test data, do not require manual intervention, automatically complete data comparison by the system, save time and effort, and have high reliability. Therefore, various problems existing in current automated testing can be solved with the least cost, the most interface problems can be discovered with the least time consumption, and flexible configuration can adapt to multiple interface test scenarios.

[0090] Figure 6 It is the structural diagram of the automated test device according to the embodiment of the present invention. Figure 6 The device shown is only an example and should not impose any limitation on the functions and usage scope of the embodiments of the present invention.

[0091] Reference Figure 6, the device includes a processor 601, a memory 602, and an input / output device 603 connected via a bus. The memory 602 includes a read-only memory (ROM) and a random access memory (RAM). Various computer instructions and data required for executing system functions are stored in the memory 602. The processor 601 reads various computer instructions from the memory 602 to perform various appropriate actions and processes. The input / output device includes an input part such as a keyboard and a mouse; an output part including, for example, a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker; a storage part including a hard disk, etc.; and a communication part including a network interface card such as a LAN card and a modem. The memory 602 also stores the following computer instructions to complete the operations specified by the device of the embodiment of the present invention: generating a plurality of test cases based on the interface call logs in the production environment, each test case including an interface name, an input parameter value, and a first output parameter value; for each test case, performing an interface call based on the input parameter value and the interface name, and obtaining the returned second output parameter value, comparing the first output parameter value with the second output parameter value, and determining whether each test case passes according to the comparison result.

[0092] Correspondingly, an embodiment of the present invention provides a computer-readable storage medium storing computer instructions, and when the computer instructions are executed, the operations specified by the above method are implemented.

[0093] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture frameworks, functions, and operations of the system, method, and device of the embodiments of the present invention. The boxes in the flowcharts and block diagrams may represent a module, a program segment, or just a piece of code, and the module, program segment, and code are all executable instructions for implementing the specified logical functions. It should also be noted that the executable instructions for implementing the specified logical functions can be recombined to generate new modules and program segments. Therefore, the boxes in the accompanying drawings and the order of the boxes are only used to better illustrate the processes and steps of the embodiments, and should not be used as a limitation to the invention itself.

[0094] Each module or unit of the system can be implemented by hardware, firmware, or software. The software includes, for example, coding programs formed in various programming languages such as JAVA, C / C++ / C#, and SQL. Although the steps and the order of the steps of the embodiments of the present invention are given in the method and the method illustration, the executable instructions for implementing the specified logical functions of the steps can be recombined to generate new steps. The order of the steps should not be limited only to the order of the steps in the method and the method illustration, and can be adjusted at any time according to the needs of the function. For example, some steps characterized therein can be executed in parallel or in the reverse order.

[0095] The systems and methods according to the present invention can be deployed on a single or multiple servers. For example, different modules can be separately deployed on different servers to form dedicated servers. Alternatively, the same functional units, modules or systems can be distributedly deployed on multiple servers to relieve the load pressure. The servers include, but are not limited to, multiple PCs, PC servers, blade servers, supercomputers, etc. within the same local area network and connected via the Internet.

[0096] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention can have various modifications and changes. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. An automated testing method, characterized in that, including: generating a plurality of test cases based on interface call logs in a production environment, each test case including an interface name, an input parameter value, and a first output parameter value; for each test case, performing an interface call based on the input parameter value and the interface name, obtaining a returned second output parameter value, comparing the first output parameter value with the second output parameter value, and determining whether each test case passes according to the comparison result; wherein the performing the interface call includes: setting up an interface application that receives the corresponding input parameter value and interface name, and calls the corresponding interface according to the corresponding input parameter and interface name, wherein the interface application is connected to the database in the production environment, and the interfaces are divided into read interfaces and write interfaces, and the database write operation is masked in the write interface, and the interface is changed to the online official alias to achieve the purpose of calling the interface in the production environment.

2. The automated testing method according to claim 1, characterized in that, further including: storing the plurality of test cases in a test library.

3. The automated testing method according to claim 1, characterized in that, further including: maintaining at least one test case through a configuration file.

4. The automated testing method according to claim 3, characterized in that, wherein the performing an interface call for each test case based on the corresponding input parameter value and interface name includes: sequentially reading each test case from the configuration file to obtain the interface name, reading the corresponding input parameter value from the test library according to the interface name, and then performing the interface call.

5. The automated testing method according to claim 1, characterized in that, the interface name includes a class name and a method name.

6. The automated testing method according to claim 1, characterized in that, wherein the generating a plurality of test cases includes: for each interface call log, determining whether the interface call log includes an output parameter value and / or an input parameter value; when an interface call log includes an output parameter value and an input parameter value, generating a test case based on the interface call log; if an interface call log only includes an output parameter value, generating a test case if an entry with an input parameter value is found in the corresponding record.

7. An automated testing system, characterized in that, including: a test case generation module for generating a plurality of test cases based on interface call logs in a production environment, each test case including an interface name, an input parameter value, and a first output parameter value; an automated testing module for, for each test case, sending the input parameter value and the interface name to the interface application, obtaining a returned second output parameter value, comparing the first output parameter value with the second output parameter value, and determining whether each test case passes according to the comparison result; an interface application for receiving the input parameter value and the interface name from the automated testing module and performing an interface call, returning the second output parameter value of the interface call to the automated testing module, wherein the interface application is connected to the database in the production environment, and for the write interface called by the interface application, the database write operation is masked in the write interface, and the read interface is changed to the online official alias to achieve the purpose of calling the interface in the production environment.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed, implement the automated testing method according to any one of claims 1 to 6.

9. An automated testing device, characterized in that, including: a memory for storing computer instructions; A processor, coupled to the memory, the processor being configured to execute an automated test method as claimed in any one of claims 1 to 6 based on computer instructions stored in the memory.

Images