Software testing system and method, electronic equipment and storage medium

Abstract

The invention relates to a software testing system and method, electronic equipment and a storage medium, and the software testing system comprises an operation parameter analysis module which obtains and analyzes test operation parameters of to-be-tested software, and determines a test case configuration file of at least one test case according to the test operation parameters; the case initialization module is used for generating at least one test case according to the test case configuration file and a first preset template; the case operation module is used for executing the test case to obtain an execution result of the test case; and the data verification module verifies the execution result of the test case to obtain a function test result of the to-be-tested software. The invention provides a highly modularized and automatic software testing system, the generation process of the test case is simplified by introducing the predefined template, a tester only needs to maintain the test operation parameters and the test case configuration file, repeated labor is reduced, the test requirements of different software and requirements can be met, and the test efficiency is improved. The universality of the system is enhanced, and the test efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a software testing system, method, electronic device and storage medium. Background Art

[0002] With the increasing popularity and complexity of software applications, the importance of software quality assurance has become increasingly prominent. Software testing, as a key link in the software development life cycle, is directly related to the reliability, stability and user experience of software products. Usually, during the functional testing and acceptance testing phases of software, testers mainly focus on the functions and performance of software products, and rarely involve their internal implementation details. Within limited resources and time, how to efficiently complete functional testing, provide detailed test reports in a timely manner, and quickly feedback test results is crucial to ensuring product quality and project progress.

[0003] However, due to the diversification of software applications and the complexity of functional testing, traditional manual testing methods can no longer meet the needs of modern software development. Although automated testing based on specific software versions and specific requirements has begun to be applied, its flexibility and intelligence still need to be improved. In order to further improve testing efficiency and quality, a more intelligent and efficient software testing system is needed to meet these challenges. Summary of the invention

[0004] In response to the above technical problems, the present application provides a highly modular and automated software testing system, method, electronic device and storage medium. By introducing predefined templates, the test case generation process is simplified. Testers only need to maintain test operation parameters and test case configuration files, reducing repetitive work, meeting the testing requirements of different software and needs, enhancing the versatility of the system, and improving testing efficiency.

[0005] In order to solve the above technical problems, the present application provides a software testing system, the system comprising:

[0006] A running parameter parsing module is used to obtain and parse the test running parameters of the software to be tested, and a test case configuration file of at least one test case is determined according to the test running parameters;

[0007] A test case initialization module generates at least one test case according to a test case configuration file and a first preset template;

[0008] The test case running module executes the test case and obtains the execution result of the test case;

[0009] The data verification module verifies the execution results of the test cases to obtain the functional test results of the software under test.

[0010] In one embodiment, the running parameter parsing module is further used to determine the test running mode of the test case according to the test running parameters, and the test running mode includes a local mode and a cluster mode;

[0011] The test case running module is also used to execute test cases according to the test running mode.

[0012] In one embodiment, the running parameter parsing module obtains and parses the test running parameters of the software to be tested, including:

[0013] Parse the test run parameters to obtain a test case configuration file generated according to user input, the test case configuration file includes configuration information of at least one test case, and the configuration information of each test case includes input file path, output file path, standard file path, and test instruction interface.

[0014] In one embodiment, the first preset template includes a use case script template, a use case execution template, and a use case decision template, and generating at least one test case according to the test case configuration file and the first preset template includes:

[0015] Get the input file path and test instruction interface in the configuration information;

[0016] Generate a use case script file based on the input file path, test instruction interface and use case script template;

[0017] Determine at least one execution statement according to the use case execution template, encapsulate the at least one execution statement to generate a use case execution file, the execution statement at least includes an identification of the software to be tested and a use case script file path determined according to the generated use case script file;

[0018] A use case decision file is generated according to the output file path, standard file path and use case decision template obtained by executing the test case. The use case decision file is used to verify the execution result of the test case.

[0019] In one embodiment, the test case running module executes the test case and obtains the execution result of the test case, including:

[0020] Execute at least one execution statement of the test case execution file, execute the test case script file according to the test case script file path in the execution statement to obtain the execution result of the test case, and store the execution result to the output file according to the output file path.

[0021] In one embodiment, the data verification module verifies the execution result of the test case, including:

[0022] Obtain the execution results of the test cases in the output file according to the output file path, and obtain the expected results of the standard file according to the standard file path;

[0023] Execute the judgment statements in the test case judgment file, and obtain the functional test results of the software to be tested based on the execution results of the test case and the expected results of the standard file.

[0024] In one embodiment, the software testing system further includes:

[0025] The performance acquisition module is used to obtain performance data during the execution of the test case, and evaluate the performance of the software under test based on the performance data to obtain the performance test results of the software under test.

[0026] The present application also provides a software testing method, comprising the following steps:

[0027] Acquire and parse the test running parameters of the software to be tested, and determine a test case configuration file of at least one test case according to the test running parameters;

[0028] Generate at least one test case according to the test case configuration file and the first preset template;

[0029] Run the test file to execute the test case according to the test run parameters;

[0030] Execute the test case and obtain the execution result of the test case;

[0031] Verify the execution results of the test cases to obtain the functional test results of the software under test.

[0032] In one embodiment, obtaining and parsing the test running parameters of the software to be tested includes:

[0033] Parse the test run parameters to obtain a test case configuration file generated according to user input, the test case configuration file includes configuration information of at least one test case, and the configuration information of each test case includes input file path, output file path, standard file path, and test instruction interface.

[0034] In one embodiment, the first preset template includes a use case script template, a use case execution template, and a use case decision template, and generates at least one test case according to the test running parameters and the preset template, including:

[0035] Get the input file path and test instruction interface in the configuration information;

[0036] Generate a use case script file based on the input file path, test instruction interface and use case script template;

[0037] Determine at least one execution statement according to the use case execution template, encapsulate the at least one execution statement to generate a use case execution file, the execution statement at least includes an identification of the software to be tested and a use case script file path determined according to the generated use case script file;

[0038] A use case decision file is generated according to the output file path, standard file path and use case decision template obtained by executing the test case. The use case decision file is used to verify the execution result of the test case.

[0039] The present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned software testing method when executing the computer program.

[0040] The present application also provides a storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the software testing method described above are implemented.

[0041] The software testing system, method, electronic device and storage medium of the present application, the software testing system includes: an operation parameter parsing module, which obtains and parses the test operation parameters of the software to be tested, and determines the test case configuration file of at least one test case according to the test operation parameters; a test case initialization module, which generates at least one test case according to the test case configuration file and a first preset template; a test case running module, which executes the test case and obtains the execution result of the test case; a data verification module, which verifies the execution result of the test case to obtain the functional test result of the software to be tested. The present application provides a highly modular and automated software testing system, which simplifies the test case generation process by introducing predefined templates. Testers only need to maintain the test operation parameters and the test case configuration file, which reduces duplication of work, can meet the test requirements of different software and needs, enhances the versatility of the system, and improves the test efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 The figure is a flowchart of a software testing system according to an embodiment.

[0043] Figure 2 It is a schematic diagram of the structure of test operation parameters according to an embodiment.

[0044] Figure 3 The figure is a schematic diagram of a test case configuration file according to an embodiment.

[0045] Figure 4 is a schematic diagram of a test case according to an embodiment.

[0046] Figure 5 The figure is a schematic diagram of a use case determination file according to an embodiment.

[0047] Figure 6 It is a flowchart of a software testing method according to an embodiment.

[0048] Figure 7It is a schematic structural diagram of an electronic device according to an embodiment. DETAILED DESCRIPTION

[0049] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0050] It should be noted that, in this article, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element. In addition, components, features, and elements with the same name in different embodiments of the present application may have the same meaning or different meanings, and their specific meanings need to be determined by their explanation in the specific embodiment or further combined with the context of the specific embodiment.

[0051] It should be understood that, although the terms first, second, third, etc. may be used to describe various information in this article, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this article, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein can be interpreted as "at the time of..." or "when..." or "in response to determination". Furthermore, as used in this article, the singular forms "one", "one" and "the" are intended to also include plural forms, unless there is an opposite indication in the context. It should be further understood that the terms "comprising", "including" indicate that there are described features, steps, operations, elements, components, projects, kinds, and / or groups, but do not exclude the existence, occurrence or addition of one or more other features, steps, operations, elements, components, projects, kinds, and / or groups. The terms "or" and "and / or" used herein are interpreted as inclusive, or mean any one or any combination. Thus, “A, B, or C” or “A, B and / or C” means “any of the following: A; B; C; A and B; A and C; B and C; A, B, and C.” An exception to this definition will occur only when a combination of elements, functions, steps, or operations are inherently mutually exclusive in some manner.

[0052] It should be understood that, although the various steps in the flowchart in the embodiment of the present application are displayed in sequence according to the indication of the arrows, these steps are not necessarily performed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and it can be performed in other orders. Moreover, at least a portion of the steps in the figure may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and their execution order is not necessarily performed in sequence, but can be performed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0053] It should be noted that in this article, step codes such as S1 and S2 are used for the purpose of expressing the corresponding content more clearly and concisely, and do not constitute a substantial limitation on the order. When implementing the step, those skilled in the art may execute S2 first and then S1, etc., but these should all be within the scope of protection of this application.

[0054] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0055] In the subsequent description, the suffixes such as "module", "component" or "unit" used to represent elements are only used to facilitate the description of the present application and have no specific meaning. Therefore, "module", "component" or "unit" can be used in a mixed manner.

[0056] First embodiment

[0057] Please refer to Figure 1 To solve the above technical problems, the present application provides a software testing system, which at least includes an operation parameter parsing module 20, a use case initialization module 21, a use case operation module 22 and a data verification module 23. Specifically,

[0058] The running parameter parsing module 20 is used to obtain and parse the test running parameters of the software to be tested, and determine the test case configuration file of at least one test case according to the test running parameters.

[0059] Among them, the test running parameters are the parameters used in the software testing system to configure the test environment and control the test execution. Figure 2 As shown, the test running parameters include but are not limited to the software to be tested identifier, the test case configuration file identifier, the test case running mode identifier and the parallel number identifier. The path or specific content of the file or information referred to by the identifier can be determined based on the identifier.

[0060] The software to be tested refers to the software product or application that is about to be tested. The software to be tested identifier in the test run parameters can be parameters such as the name and / or path of the software to be tested. The software testing system can identify and call the software to be tested based on the software to be tested identifier. The test case configuration file refers to the file required to generate, execute and verify the test case. The test case configuration file identifier can be parameters such as the name and / or path of the test case configuration file. The specific information in the test case configuration file can be found based on the name and / or path. The test case running mode identifier is used to obtain the execution mode of the test case, such as the local mode or the cluster mode. The parallel number identifier is used to obtain the parallel number. The parallel number refers to the number of test cases that can be executed concurrently in the same time period.

[0061] During the system operation, the operation parameter parsing module 20 is responsible for parsing the content of the test operation parameters, and then passing the variables corresponding to the test operation parameters to other modules in the software testing system. Specifically, the operation parameter parsing module 20 first obtains the test operation parameters, extracts and records part or all of the content in the test operation parameters as variables. Then, the system calls the variables to implement the generation, execution, verification and other processes of the test case. The test operation parameters are obtained in a variety of ways including but not limited to reading from a file, reading from a database, receiving data transmitted by a user interface, and obtaining from an external service or API interface.

[0062] The following is an example of a set of test run parameters:

[0063] qahub-L software-LC testcase_file-FM local-MP 10

[0064] Among them, qahub is the name of the software testing system; “–L software” indicates the software to be tested, and software is the name of the software to be tested; “–LC testcase_file” indicates the test case configuration file identifier, and testcase_file is the name of the test case configuration file; “–FMlocal” indicates the test case running mode identifier, and local indicates that the test case running mode is local; “–MP 10” indicates the parallel number identifier, and 10 indicates that the maximum number of test cases that the software testing system supports to be executed simultaneously is 10.

[0065] The use case initialization module 21 generates at least one test case according to the test case configuration file and the first preset template, and the test case is used to test the software to be tested.

[0066] Among them, the first preset template is a test case template containing placeholders that is pre-defined by developers or testers in the software testing system, and is used to specify the basic structure of the test case. The operation parameter parsing module extracts corresponding variables from the test operation parameters based on the identifier, such as "extracting the name of the software to be tested based on the software to be tested identifier", "extracting the test case configuration file based on the test case configuration file identifier", etc. The case initialization module 21 fills these variables into the corresponding placeholders of the first preset template, thereby automatically generating a test case. Different test operation parameters can generate a variety of test cases, which improves test efficiency and flexibility. The variables extracted from the test operation parameters can be the parameters themselves, such as "the name of the software to be tested", or the specific content of the parameters, such as "the name or path of each test case input and output file in the test case configuration file". In this way, the test case generation process is simplified, and the tester only needs to maintain the test operation parameters and the test case configuration file, reducing duplication of work.

[0067] The test case running module 22 executes the test case and obtains the execution result of the test case.

[0068] The test case running module 22 obtains the test case from the test case initialization module 21. The test environment is prepared according to the configuration of the test case, and then the test case is executed and the execution results of the test case are collected.

[0069] The data verification module 23 verifies the execution result of the test case to obtain the functional test result of the software to be tested.

[0070] The data verification module 23 receives the execution result of the test case from the test case running module 22, compares the execution result with the expected result, and draws a conclusion on whether the test case passes or not.

[0071] The software testing system of the embodiment of the present application is based on black box testing technology, which automatically generates test cases through preset templates and test run parameters without in-depth understanding of the internal implementation of the software. It can solve the technical problems of low efficiency of manual testing and insufficient flexibility of automated testing based on specific software versions and specific requirements.

[0072] It should be noted that in the field of software testing, black box testing is also called functional testing. It is a software testing method that focuses on the functions and outputs of the software without considering its internal implementation and structure. When conducting black box testing, testers only need to understand the specifications and functional requirements of the software without having to understand its source code or implementation details. The core of black box testing is to verify whether the software performs as expected and meets the user's needs and specifications. By using black box testing methods reasonably, the quality and reliability of the software can be effectively improved.

[0073] In one embodiment, the running parameter parsing module 20 is further used to determine the test running mode of the test case according to the test running parameters, and the test running mode includes a local mode and a cluster mode;

[0074] The test case running module 22 is also used to execute the test case according to the test running mode.

[0075] As an implementation method of this embodiment, when the use case initialization module 21 generates multiple test cases, the use case running module 22 obtains the test case running mode in the test running parameters. The test case running mode is used to define the execution mode of the test case, including local mode or cluster mode. The local mode means that the test case is executed on a single computing device. The cluster mode means that the test case is synchronously and concurrently executed on multiple computing devices (such as virtual machines).

[0076] Both the local mode and the cluster mode support sequential execution and / or concurrent execution of multiple test cases. During sequential execution, the test cases are executed in sequence, and the next test case is executed after each test case is executed. Sequential execution is applicable to situations where there is an interactive relationship between multiple test cases. For example, the output of test case A is used as the input of test case B, then set test case B to be executed after test case A is executed. During concurrent execution, multiple test cases can run at the same time to improve test efficiency. The parallel number refers to the number of test cases that can be executed concurrently in the same time period. Of course, the software testing system also supports a hybrid mode of sequential and concurrent operation, which can be flexibly configured according to test requirements. In this way, the software testing system of the embodiment of the present application supports the concurrent execution of test cases generated by multiple devices and multiple threads, thereby improving test efficiency.

[0077] In one embodiment, the running parameter parsing module obtains and parses the test running parameters of the software to be tested, including:

[0078] Parse the test run parameters to obtain a test case configuration file generated according to user input, the test case configuration file includes configuration information of at least one test case, and the configuration information of each test case includes input file path, output file path, standard file path, and test instruction interface.

[0079] Configuration information refers to the parameters used to obtain the files required in the process of test case generation, execution and result verification, including input file path, output file path, standard file path and test instruction interface, etc., to ensure that the test process can be carried out accurately according to the predetermined requirements. Among them, the input file path is the location of the file that stores the input data required for the test case. The output file path refers to the location of the file that stores the output data after the test case is executed. The standard file path refers to the location of the standard file used to judge the test results. The test instruction interface is a way to obtain test instructions.

[0080] As an implementation of this embodiment, the software testing system of this embodiment also includes a second preset template. The second preset template is used to define the composition format of the configuration information corresponding to each test case, so that the system can smoothly extract the configuration information corresponding to the test case based on the second preset template. In addition, the user can input the configuration information of the test case according to the second preset template.

[0081] In the second preset template, the test case name, such as TestCase1, and variable names, such as input_file (input file), run_cmd (test command interface), output_file (output file) and golden_file (standard file) are defined. The corresponding placeholder variable (i.e. the content of the variable) can be set after each variable name. Figure 3 As shown, multiple test cases such as TestCase1, TestCase2 and TestCase3 are included, and each test case includes variable names such as input_file (input file), run_cmd (test instruction interface), output_file (output file) and golden_file (standard file) and corresponding actual values. The actual value can be the name or path of the configuration file. Specifically, the input file path or name in TestCase1 is file_1, the test instruction interface is cmd_1, the output file path or name is log_1, and the standard file path or name is log_g1. The input file path or name in TestCase2 is file_2, the test instruction interface is cmd_2, the output file path or name is log_2, and the standard file path or name is log_g2. The input file includes the input data when testing the software to be tested, the test instruction is the execution statement for testing the software to be tested, the output file includes the output data obtained after testing the software to be tested, and the standard file is a standard file for judging the output file.

[0082] In this way, the software testing system can generate a test case configuration file through the second preset template, and automatically generate a complete test case based on the first preset template file and the test case configuration file, reducing the workload required to write test cases and improving the consistency and maintainability of test cases.

[0083] In one embodiment, the first preset template includes a use case script template, a use case execution template, and a use case decision template, and generating at least one test case according to the test case configuration file and the first preset template includes:

[0084] Get the input file path and test instruction interface in the configuration information;

[0085] Generate a use case script file based on the input file path, test instruction interface and use case script template;

[0086] Determine at least one execution statement according to the use case execution template, encapsulate the at least one execution statement to generate a use case execution file, the execution statement at least includes an identification of the software to be tested and a use case script file path determined according to the generated use case script file;

[0087] A use case decision file is generated according to the output file path, standard file path and use case decision template obtained by executing the test case. The use case decision file is used to verify the execution result of the test case.

[0088] In one embodiment, the input file path or name is determined according to the identifier of the input file in the configuration information, and the input file can be obtained according to the input file path or name, thereby obtaining the information in the input file. The test instruction can be obtained according to the test instruction interface. According to the use case script template, the input file path or name is combined with the test instruction to generate a use case script file.

[0089] As an implementation method of this embodiment, the test case consists of a use case script file, a use case execution file, a use case judgment file, a use case log file and a use case post-processing file.

[0090] like Figure 4 In the test case shown, the test case script file script.py at least includes the input file path or name input-file and the test command run-cmd. The process of generating the test case script file is as follows: obtain the input file path and the test command interface in the configuration information. Fill in the input file path and the test command in the preset test case script template to obtain the test case script file.

[0091] The use case execution file run.sh includes at least one execution statement, a software identification to be tested, a use case script file and a post-processing program post-process. The generation process of the use case execution file is: determine at least one execution statement according to the use case execution template, the execution statement includes at least a test execution statement and a judgment execution statement, the test execution statement is used to execute the test process, the judgment execution statement is used to execute the judgment process, the test execution statement content includes the software identification to be tested, the use case script file, etc., the judgment execution statement includes the corresponding post-processing program identification for judgment, etc., and the test execution statement and the judgment execution statement are encapsulated to obtain the use case execution file. Among them, the execution statement refers to the instructions and parameters used to define the test case execution logic, including but not limited to starting the software to be tested, making the software to be tested execute the use case script file and other steps. Encapsulation of the execution statement refers to packaging the instructions and parameters defining the test case execution logic into an independent executable running unit, which occupies a process during the execution process. The software testing system can run the test case by calling the use case execution file in the test environment. The use case execution script file is encapsulated into the use case execution file, so that the code of the use case running module is universal, and the software testing system can process different software under test or test cases in a universal way. The use case execution file packages the execution statements of different software under test, and the software testing system can distribute them uniformly to the computer for parallel processing.

[0092] The use case determination file is generated by the use case initialization module 21 according to the output file path, the standard file path and the use case determination template. A test case may correspond to multiple inspection items. The use case determination file contains one or more inspection items applicable to the test case, as well as the determination logic or rules for whether each inspection item passes, which is used to automatically compare the output file output-file (accessed through the output file path) and the standard file golden-file (accessed through the standard file path) of the actual test to determine whether the test is successful, so as to verify the execution result of the test case. The use case determination process can be implemented by executing the post-processing program in the use case execution file, calling the use case determination file by the post-processing program, and starting to parse multiple checkpoints of the use case determination file.

[0093] The test case log file is a log record generated during the execution of the test case, that is, the log file generated by running the test case execution file, including but not limited to the timestamps of the start and end of the test, the specific commands executed and their parameters, the output information during the operation (such as standard output, error output), the state changes during the operation, such as memory usage, CPU occupancy, etc., and abnormal or error information. The test case log file can help developers locate the problem when the test fails.

[0094] The test case post-processing file is a post-processing file generated by the post-processing program to determine whether the test case passes.

[0095] In one embodiment, the test case running module 22 executes the test case and obtains the execution result of the test case, including:

[0096] Execute at least one execution statement of the test case execution file, execute the test case script file according to the test case script file path in the execution statement to obtain the execution result of the test case, and store the execution result to the output file according to the output file path.

[0097] As an implementation method of this embodiment, the use case running module is responsible for executing the test case, calling and executing the corresponding use case script file by running the execution statement in the use case execution file, thereby obtaining the test results, and saving these results to the specified output file according to the preset output file path. This process realizes the automatic execution of the test case and the automatic recording of the results.

[0098] In one embodiment, the data verification module 23 verifies the execution result of the test case, including:

[0099] Obtain the execution results of the test cases in the output file according to the output file path, and obtain the expected results of the standard file according to the standard file path;

[0100] Execute the judgment statements in the test case judgment file, and obtain the functional test results of the software to be tested based on the execution results of the test case and the expected results of the standard file.

[0101] As an implementation method of this embodiment, functional testing refers to verifying whether the function of the software to be tested is correctly implemented according to the requirements specification, that is, whether the software can complete the expected tasks and whether these tasks are completed in the expected manner. According to the output file path specified in the configuration information, the output file generated after the test case is executed is obtained, and the execution result of the test case in the output file is read. At the same time, according to the standard file path specified in the configuration information, the standard file is obtained, and the expected test output result in the standard file is read. The judgment statement in the execution case judgment file is used to compare the contents of the output file and the standard file to obtain the judgment result. The judgment statement may include one or more checkpoints, each checkpoint corresponds to a check function, which is used to evaluate a specific aspect of the output file. The check function determines whether the test case passes by comparing the output file and the standard file. For example, check whether certain specific patterns or data points in the output file match the expected values ​​in the standard file. Different check functions can be used for processing for different data patterns or error conditions to ensure that the software function is verified from multiple angles.

[0102] like Figure 5As shown in the figure, the specific content of the use case judgment file is displayed, including multiple CheckPoints. Each checkpoint corresponds to a check function, which is used to evaluate the execution result of the test case. The check function generates the judgment result by processing the output file and the standard file. For CheckPoint1, the detection function check_function_1 is used to process the output file f1.log and the standard file g1.log. For CheckPoint2, the detection function check_function_2 is used to process the output file f1.log and the standard file g1.log. Different checkpoints can use different inspection standards or methods to verify different aspects of the same output file. Through multi-angle inspection, the test requirements can be covered more comprehensively.

[0103] The data verification module 23 automatically obtains the test case execution results in the output file (accessed through the output file path) and the expected results in the standard file (accessed through the standard file path) by executing the use case judgment file, and compares the two according to the judgment statement in the judgment file, so as to determine whether the functional test results of the software to be tested meet expectations. This process ensures the accuracy and consistency of the test results.

[0104] In one embodiment, the software testing system further includes:

[0105] The performance collection module 24 is used to obtain performance data during the execution of the test case, and evaluate the performance of the software to be tested based on the performance data to obtain a performance test result of the software to be tested.

[0106] As an implementation of this embodiment, performance data refers to various indicators and measured values ​​about software performance collected during the testing process of the software to be tested. Performance testing refers to evaluating performance indicators such as running time and running memory of the software to be tested based on performance data.

[0107] Performance data may include running time, running memory, error rate, resource utilization, etc. Running time refers to the time required for the software under test to complete the test case. Running memory refers to the amount of memory occupied by the software under test during operation. Error rate refers to the proportion of requests that have errors within a certain period of time. Resource utilization refers to the use of system resources (such as CPU, memory, disk I / O, network bandwidth). The performance acquisition module 24 module generates detailed performance test results by analyzing performance data, which may include a summary of performance data, analysis results, performance evaluation conclusions, and improvement suggestions, which are used to identify performance bottlenecks in the software under test for optimization.

[0108] In one embodiment, in response to changes in the software to be tested, the test running parameters and / or the first preset template are updated.

[0109] As an implementation method of this embodiment, when the requirements of the software to be tested change, such as adding new functions or removing old functions, it is only necessary to update the test running parameters and add or delete relevant test items in the test case configuration file accordingly, avoiding large-scale changes to other files or modules in the system.

[0110] As an implementation of this embodiment, when the software to be tested changes, such as upgrading from one version to another, or replacing completely different software, it is necessary to update the test running parameters and the first preset template to generate new test cases.

[0111] In one embodiment, the software testing system further includes a report display module 25 for counting and / or displaying the execution results of the test cases.

[0112] As an implementation method of this embodiment, the report display module is used to count the execution results of all test cases, and generate a test report in any readable form with the statistical test results to display to the user. The test report can be in the form of a detailed list, listing each test case and its results, or using a chart to intuitively display the test results, such as a pie chart to represent the pass rate and failure rate, a bar chart to represent the test coverage of each module, etc. The test report generation process is automated, and there is no need for manual data collation or report writing. This improves generation efficiency and reduces the possibility of human error.

[0113] In the software testing system of the embodiment of the present application, the operation parameter parsing module 20, the use case operation module 22, the performance acquisition module 24 and the report display module 25 are not related to the software to be tested, and can be reused in multiple projects, which significantly improves work efficiency. For example, a performance acquisition module can be used to monitor the response time, memory usage, etc. of different applications. Since the above modules are universal, the development team does not have to write similar codes from scratch every time they face new software to be tested, which makes the construction and configuration of the software testing system faster and greatly reduces development time and cost.

[0114] The software testing system of the embodiment of the present application includes: an operation parameter parsing module, which obtains and parses the test operation parameters of the software to be tested, and determines the test case configuration file of at least one test case according to the test operation parameters; a test case initialization module, which generates at least one test case according to the test case configuration file and a first preset template; a test case operation module, which executes the test case and obtains the execution result of the test case; and a data verification module, which verifies the execution result of the test case. The present application provides a highly modular and automated software testing system, which simplifies the test case generation process by introducing predefined templates. Testers only need to maintain the test operation parameters and the test case configuration file, which reduces duplication of work, can meet the test requirements of different software and needs, enhances the versatility of the system, and improves test efficiency.

[0115] Second embodiment

[0116] See also Figure 6 The embodiment of the present application also provides a software testing method, which can be implemented in software and / or hardware, such as a computer, server or other computing device. The software testing method provided in this embodiment includes:

[0117] S1. Obtain and parse the test running parameters of the software to be tested, and determine a test case configuration file of at least one test case according to the test running parameters.

[0118] S2. Generate at least one test case according to the test case configuration file and the first preset template, where the test case is used to test the software to be tested.

[0119] S3. Run the test file according to the test run parameters to execute the test case.

[0120] S4. Execute the test case and obtain the execution result of the test case.

[0121] S5. Verify the execution results of the test cases to obtain the functional test results of the software under test.

[0122] In one embodiment, the running parameter parsing module is further used to determine the test running mode of the test case according to the test running parameters, and the test running mode includes a local mode and a cluster mode;

[0123] The test case running module is also used to execute test cases according to the test running mode.

[0124] In one embodiment, obtaining and parsing the test running parameters of the software to be tested includes:

[0125] Parse the test run parameters to obtain a test case configuration file generated according to user input, the test case configuration file includes configuration information of at least one test case, and the configuration information of each test case includes input file path, output file path, standard file path, and test instruction interface.

[0126] In one embodiment, the first preset template includes a use case script template, a use case execution template, and a use case decision template, and generates at least one test case according to the test running parameters and the preset template, including:

[0127] Get the input file path and test instruction interface in the configuration information;

[0128] Generate a use case script file based on the input file path, test instruction interface and use case script template;

[0129] Determine at least one execution statement according to the use case execution template, encapsulate the at least one execution statement to generate a use case execution file, the execution statement at least includes an identification of the software to be tested and a use case script file path determined according to the generated use case script file;

[0130] A use case decision file is generated according to the output file path, standard file path and use case decision template obtained by executing the test case. The use case decision file is used to verify the execution result of the test case.

[0131] In one embodiment, verifying the execution result of the test case includes:

[0132] Obtain the use case judgment file and the execution results of the test case;

[0133] Evaluate the execution results of the test cases according to the use case judgment file to obtain the functional test results of the software under test.

[0134] In one embodiment, verifying the execution result of the test case includes:

[0135] Obtain the execution results of the test cases in the output file according to the output file path, and obtain the expected results of the standard file according to the standard file path;

[0136] Execute the judgment statements in the test case judgment file, and obtain the functional test results of the software to be tested based on the execution results of the test case and the expected results of the standard file.

[0137] In one embodiment, the software testing system further includes:

[0138] The performance acquisition module is used to obtain performance data during the execution of the test case, and evaluate the performance of the software under test based on the performance data to obtain the performance test results of the software under test.

[0139] In one embodiment, the software testing method further includes:

[0140] In response to changes in the software to be tested, the test running parameters and / or the first preset template are updated.

[0141] For the specific implementation steps of the software testing method in the embodiments of the present application, please refer to the relevant description of the software testing system in the first embodiment, which will not be repeated here.

[0142] Based on the same inventive concept as the above embodiments, an embodiment of the present invention provides an electronic device, such as Figure 7 As shown, the electronic device includes: a processor 310 and a memory 311 storing a computer program; wherein, Figure 7 The processor 310 shown in the figure is not used to indicate that the number of the processor 310 is one, but is only used to indicate the positional relationship of the processor 310 relative to other devices. In actual applications, the number of the processor 310 may be one or more; similarly, Figure 7 The memory 311 shown in the figure has the same meaning, that is, it is only used to refer to the position relationship of the memory 311 relative to other devices. In practical applications, the number of memories 311 can be one or more. When the processor 310 runs the computer program, the software testing method applied to the above electronic device is implemented.

[0143] The electronic device may also include: at least one network interface 312. The various components in the electronic device are coupled together via a bus system 313. It is understood that the bus system 313 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 313 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, the bus system 313 is not used in the following examples. Figure 7 Various buses are labeled as bus system 313.

[0144] The memory 311 may be a volatile memory or a nonvolatile memory, or may include both volatile and nonvolatile memories. The nonvolatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disk, or a compact disc read-only memory (CD-ROM); the magnetic surface memory may be a disk memory or a tape memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SDRAM), and so on.

[0145] (SLDRAM, SyncLink Dynamic Random Access Memory), Direct Rambus Random Access Memory (DRRAM, Direct Rambus Random Access Memory). The memory 311 described in the embodiment of the present invention is intended to include but is not limited to these and any other suitable types of memory.

[0146] The memory 311 in the embodiment of the present invention is used to store various types of data to support the operation of the electronic device. Examples of these data include: any computer program used to operate on the electronic device, such as an operating system and an application; contact data; phone book data; messages; pictures; videos, etc. Among them, the operating system includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., which are used to implement various basic services and process hardware-based tasks. The application program may include various applications, such as a media player (Media Player), a browser (Browser), etc., which are used to implement various application services. Here, the program that implements the method of the embodiment of the present invention may be included in the application program.

[0147] Based on the same inventive concept as the above-mentioned embodiment, this embodiment further provides a storage medium, in which a computer program is stored, and the storage medium may be a magnetic random access memory (FRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a magnetic surface memory, an optical disk, or a compact disc read-only memory (CD-ROM) or the like; or may be various devices including one or any combination of the above-mentioned memories, such as a mobile phone, a computer, a tablet device, a personal digital assistant, etc. When the computer program stored in the storage medium is run by the processor, the above-mentioned software testing method is implemented. For the specific steps and processes implemented when the computer program is executed by the processor, please refer to the description of the above-mentioned embodiment, which will not be repeated here.

[0148] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0149] In this document, the terms "comprises," "comprising," or any other variations thereof, are intended to cover a non-exclusive inclusion of elements other than those listed and may also include additional elements not expressly listed.

[0150] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A software testing system, characterized in that: The system comprises: A running parameter parsing module is used to obtain and parse the test running parameters of the software to be tested, and determine a test case configuration file of at least one test case according to the test running parameters; A test case initialization module, generating at least one test case according to the test case configuration file and the first preset template; A test case running module executes the test case and obtains the execution result of the test case; The data verification module verifies the execution result of the test case to obtain the functional test result of the software to be tested.

2. The system according to claim 1, characterized in that The running parameter parsing module is further used to determine the test running mode of the test case according to the test running parameters, and the test running mode includes a local mode and a cluster mode; The test case running module is also used to execute the test case according to the test running mode.

3. The system according to claim 1, characterized in that The operation parameter parsing module obtains and parses the test operation parameters of the software to be tested, including: Parse the test running parameters to obtain a test case configuration file generated according to user input, wherein the test case configuration file includes configuration information of at least one test case, and the configuration information of each test case includes input and output file paths, standard file paths, and a test instruction interface.

4. The system according to claim 3, characterized in that The first preset template includes a use case script template, a use case execution template and a use case determination template, and the use case initialization module generates at least one test case according to the test case configuration file and the first preset template, including: Obtain the input file path and test instruction interface in the configuration information; Generate a use case script file according to the input file path, the test instruction interface and the use case script template; Determine at least one execution statement according to the use case execution template, encapsulate the at least one execution statement to generate a use case execution file, wherein the execution statement at least includes an identification of the software to be tested and a use case script file path determined according to the generated use case script file; A use case determination file is generated according to the output file path obtained by executing the test case, the standard file path and the use case determination template, and the use case determination file is used to verify the execution result of the test case.

5. The system according to claim 4, characterized in that The test case running module executes the test case and obtains the execution result of the test case, including: Execute at least one execution statement of the use case execution file, execute the use case script file according to the path of the use case script file in the execution statement to obtain the execution result of the test case, and store the execution result to the output file according to the output file path.

6. The system according to claim 4, characterized in that The data verification module verifies the execution result of the test case, including: Obtaining the execution result of the test case in the output file according to the output file path, and obtaining the expected result of the standard file according to the path of the standard file; Execute the judgment statements in the test case judgment file, and obtain the functional test results of the software to be tested according to the execution results of the test case and the expected results of the standard file.

7. The system according to claim 1, characterized in that The system further comprises: The performance acquisition module is used to obtain performance data during the execution of the test case, and evaluate the performance of the software to be tested based on the performance data to obtain a performance test result of the software to be tested.

8. A software testing method, characterized in that: The method comprises the following steps: Acquire and parse the test running parameters of the software to be tested, and determine a test case configuration file of at least one test case according to the test running parameters; Generate at least one test case according to the test case configuration file and the first preset template; Run the test file according to the test run parameters to execute the test case; Execute the test case to obtain the execution result of the test case; Verify the execution result of the test case to obtain the functional test result of the software to be tested.

9. The method according to claim 8, characterized in that The obtaining and parsing of the test running parameters of the software to be tested includes: Parse the test running parameters to obtain a test case configuration file generated according to user input, wherein the test case configuration file includes configuration information of at least one test case, and the configuration information of each test case includes an input file path, an output file path, a standard file path, and a test instruction interface.

10. The method according to claim 9, characterized in that The first preset template includes a use case script template, a use case execution template and a use case determination template, and the generating at least one test case according to the test case configuration file and the preset template includes: Obtain the input file path and the test instruction interface in the configuration information; Generate a use case script file according to the input file path, the test instruction interface and the use case script template; Determine at least one execution statement according to the use case execution template, encapsulate the at least one execution statement to generate a use case execution file, wherein the execution statement at least includes an identification of the software to be tested and a use case script file path determined according to the generated use case script file; A use case determination file is generated according to the output file path obtained by executing the test case, the standard file path and the use case determination template, and the use case determination file is used to verify the execution result of the test case.

11. An electronic device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 8 to 10 when executing the computer program.

12. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 8 to 10 is implemented.

Images