Batch test case generation and testing method, electronic device and storage medium

Abstract

The present application relates to the field of electronic design automation testing, in particular to a batch test case generation and testing method, an electronic device and a storage medium. The method comprises: pre-configuring a test template, the configuration test template comprising a solidification part and a variable part, the variable part comprising test input variables and expected result variables, and a verification function for verifying the test results and expected results of the test case; pre-configuring a test data configuration file, the test data configuration file comprising a test case data set, the test case data set comprising a plurality of test case data; in response to unit testing, instantiating each test case data to the test template using a preset script to generate a test case, and running the test case, and packaging the test case based on the verification result of the verification function. The batch test case generation and testing method of the present application can generate test cases in batches, improving the test efficiency of unit testing.

Description

Technical Field

[0001] This invention relates to the field of automated testing in electronic design, and in particular to a method for generating and testing batch test cases, an electronic device, and a storage medium. Background Technology

[0002] White-box testing is a software testing technique, also known as structural testing or logic-driven testing. It focuses on how the unit under test functions internally. It designs test cases based on the program's control structure and is primarily used for software program verification.

[0003] Current white-box testing technologies primarily rely on manually writing test cases and performing functional testing. This manual approach requires testers to continuously type large amounts of repetitive code, leading to a heavy testing workload and low efficiency. Furthermore, this method directly results in fewer test cases, leading to incomplete test coverage, missed tests, and prolonged testing cycles. Summary of the Invention

[0004] To address the shortcomings of existing technologies, the purpose of this application is to provide a method for generating and testing batch test cases, an electronic device, and a storage medium to improve the efficiency of white-box testing.

[0005] To achieve the above objectives, this application provides a method for generating and testing batch test cases, including:

[0006] A pre-configured test template is provided, which includes a fixed part and a variable part. The variable part includes test input variables and expected result variables, as well as a verification function for verifying the test results and expected results of the test cases.

[0007] A test data configuration file is pre-configured. The test data configuration file includes a test case data set, which includes several test case data. Each test case data includes a test case name and several data elements. The data elements are divided into test data corresponding to test input variables and expected result data corresponding to expected result variables.

[0008] In response to unit tests, pre-defined scripts are used to instantiate the data of each test case into the test template to generate test cases, and the test cases are run. The test cases are then packaged based on the verification results of the verification function.

[0009] Furthermore, the specific steps for responding to unit tests, using preset scripts to instantiate each test case data into the test template to generate test cases, running the test cases, and packaging the test cases based on the verification results of the verification function include:

[0010] Parse the test data configuration file to obtain the data for each test case;

[0011] Make several copies of the test template as test cases to be assembled, and parse out the fixed part and the variable part respectively;

[0012] Replace the variables in the variable section of each test case with the data elements of each test case data to form each test case.

[0013] Furthermore, the specific steps of responding to unit tests, using preset scripts to instantiate each test case data into the test template to generate test cases, running the test cases, and packaging the test cases based on the verification results of the verification function also include:

[0014] Run each test case to generate a test case log file, parse the file, and determine whether the verification result of the verification function is correct. If incorrect, return; if correct, generate a test case compressed package.

[0015] Furthermore, the method also includes:

[0016] Check the correctness of all test case packages and count the number of failed test cases.

[0017] Furthermore, the description also includes:

[0018] Configure a common configuration file, which is used to configure the common filename of test cases so that files that do not belong to the common filename are deleted before packaging after the test cases are run.

[0019] Furthermore, the number of data elements in the test case data is consistent with the number of variables in the variable section.

[0020] To achieve the above objectives, the electronic device provided in this application includes:

[0021] processor;

[0022] A memory having stored thereon one or more computer program instructions that run on the processor;

[0023] When the processor executes the computer instructions, it performs the batch test case generation and testing method described above.

[0024] To achieve the above objectives, the present application provides a computer-readable storage medium that stores computer instructions thereon, which, when executed by a processor, perform the steps of the batch test case generation and testing method described above.

[0025] The batch test case generation and testing method of this application can reduce the manual writing operations of testers and automate the process of creating unit test cases. Testers only need to focus on the test data and expected results of the test cases.

[0026] The batch test case generation and testing method of this application allows testers to focus on test data input and expected output for certain specific interfaces. The execution results of all test cases must meet the expected output. If they do not, it means that the interface test has failed, indicating that there is a bug in the interface. The entire test case writing is very minimal, the operation process is highly automated, and the testing efficiency is high, which can improve the testing efficiency by about 5 to 10 times.

[0027] Other features and advantages of this application will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing this application. Attached Figure Description

[0028] The accompanying drawings are provided to further illustrate the present application and form part of the specification. Together with the embodiments of the present application, they serve to explain the present application but do not constitute a limitation thereof. In the drawings:

[0029] Figure 1 This is a flowchart illustrating the batch test case generation and testing method of Embodiment 1 of this application;

[0030] Figure 2 This is a schematic diagram of the test template of Embodiment 1 of this application;

[0031] Figure 3 This is a schematic diagram of the contents of the public configuration file of Embodiment 1 of this application;

[0032] Figure 4 This is a schematic diagram of the test data configuration file content of Embodiment 1 of this application;

[0033] Figure 5 This is a schematic diagram of the contents of the public path configuration file in Embodiment 1 of this application;

[0034] Figure 6 This is a schematic diagram of the contents of a test case in Embodiment 1 of this application;

[0035] Figure 7 This is a schematic diagram of the script structure of Embodiment 1 of this application. Detailed Implementation

[0036] Embodiments of this application will now be described in more detail with reference to the accompanying drawings. While some embodiments of this application are shown in the drawings, it should be understood that this application can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this application. It should be understood that the drawings and embodiments of this application are for illustrative purposes only and are not intended to limit the scope of protection of this application.

[0037] It should be understood that the steps described in the method embodiments of this application may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this application is not limited in this respect.

[0038] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "an embodiment"; the term "another embodiment" means "an additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0039] It should be noted that the terms "one" and "multiple" used in this application are illustrative rather than restrictive, and those skilled in the art should understand that, unless explicitly stated otherwise in the context, they should be understood as "one or more". "Multiple" should be understood as two or more.

[0040] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0041] Example 1

[0042] One embodiment of this application provides a method for generating and testing batch test cases, which improves the automation level of the testing process and increases testing efficiency.

[0043] Figure 1 This is a flowchart illustrating the batch test case generation and testing method of Embodiment 1 of this application. The following will refer to... Figure 1 This application provides a detailed description of the batch test case generation and testing method of Embodiment 1. Embodiment 1 uses the eng() interface of the conversion engineering counting method as an example, employing the following steps to automate the testing process and automatically generate batch test cases:

[0044] Step S101: Pre-configure the test template. The configured test template includes a fixed part and a variable part. The variable part includes test input variables and expected result variables, as well as a verification function for verifying the test results and expected results of the test cases.

[0045] Figure 2 This is a schematic diagram of the test template of Embodiment 1 of this application. (See attached diagram.) Figure 2 In the diagram, the pre-fixed code section and the post-fixed code section are the fixed parts, while {_PAR_001} and {_PAR_002} in the modular part are the variable parts, with {_PAR_001} being the test input variable and {_PAR_002} being the expected result variable.

[0046] The test template is written by the testers themselves. It includes all the fixed parts of this unit test and the variable part that requires the test data to be filled in. In addition to the test input variables for filling in the test data, the variable part also includes the expected result variables for filling in the expected results. The workload is basically the same as writing a test case in the traditional way.

[0047] In this embodiment, a common configuration file is also configured. This common configuration file is used to configure the common filenames of test cases, so that files that do not belong to the common filenames are deleted before packaging after the test cases have run. See also Figure 3 , Figure 3 This is a schematic diagram of the contents of the public configuration file in Embodiment 1 of this application. The ignore field indicates that if these files are matched in the template, they will not be deleted. The public configuration file generally includes: initialization file name, preprocessing file name, postprocessing file name, comparison file name, and other files that may be needed. When running test cases, redundant files may be generated, so it is necessary to delete other files except for these file names before packaging.

[0048] In this implementation, self.assertEqual() is a verification function used to verify whether the execution result of the test case is consistent with the expected result. This function is a unit test library function in Python, used in unit tests to check the equality of two values.

[0049] Step S102: Pre-configure the test data configuration file. The test data configuration file includes a test case data set, which includes several test case data. Each test case data includes a test case name and several data elements. The data elements are divided into test data corresponding to test input variables and expected result data corresponding to expected result variables.

[0050] Figure 4 This is a schematic diagram of the test data configuration file content of Embodiment 1 of this application. (See attached diagram) Figure 4 ,in Figure 4 The example illustrates the contents of the test data configuration file, which includes three test case data entries. Each test case data entry includes two elements corresponding to the variables {_PAR_001} and {_PAR_002}.

[0051] It is understandable that the data element corresponding to {_PAR_001} is the test input for the corresponding test case, and the data element corresponding to {_PAR_002} is the expected result for the corresponding test case.

[0052] Testers can focus their main efforts on writing test data configurations: listing the test data to be tested and the correct expected results.

[0053] Understandably, when configuring the test data configuration file, the number of data elements in the test data should be the same as the number of variables in the test template.

[0054] In this implementation, toml is used, but in actual use it is not limited to toml. Any method that can correctly represent data is acceptable.

[0055] In this embodiment, a public path configuration file is also configured. Figure 5 This is a schematic diagram of the contents of the public path configuration file in Embodiment 1 of this application. (See attached diagram.) Figure 5 It utilizes public paths to configure relevant datasets.

[0056] Step S103: In response to unit tests, use a preset script to instantiate the data of each test case into the test template to generate test cases, run the test cases, and package the test cases based on the verification results of the verification function.

[0057] When unit testing begins, a pre-defined script is used to generate individual test cases from the test template by transferring the test data for each test case. Figure 6 This is a schematic diagram of a test case in Embodiment 1 of this application, as shown below. Figure 6 As shown, the positions of the variables {_PAR_001} and {_PAR_002} in the test template have been replaced with... Figure 4 The test case data case_001 contains two elements: 1230000000 and 1.23e+09.

[0058] like Figure 6 As shown, the verification function `self.assertEqual()` verifies whether the execution result of the test case matches the expected result. If they match, the test case is automatically packaged; otherwise, it is logged. Test cases with errors logged indicate a bug in the interface, requiring notification to the development team for modification.

[0059] Figure 7 This is a schematic diagram of the script structure of Embodiment 1 of this application. (See attached diagram) Figure 7The code snippet includes the following classes: `Log` (for outputting logs during the test case generation process); `parse_case` (for parsing the test case template); `parse_toml` (for parsing the data file, obtaining replacement field parameters, and finally calling the script to run the test cases); `parse_pyrun` (for parsing and obtaining the `gold` file written in the `pyRun.sh` file, as some functions cannot be verified by unit test code, and certain special functions require comparison of screenshots, binary files, or output files, the `pyRun.sh` file stores the file types and filenames to be compared, collectively referred to as `gold` files); `mpl_exec` (for multi-threaded test case generation); `gen_run_cmd` (a source preprocessing file that prepares the environment for running the unit test script); `make_case` (the main processing logic of the test case generation program, completing template reading, dataset parsing, environment preparation, running the `py` file, checking the verification results of the verification function, deleting process files, and packaging); and `gen_case` (the main entry point for the test case generation program).

[0060] The script execution process is as follows:

[0061] When the script is run, it first enters the main entry point gen_case. This function calls parse_case and adds exception handling to ensure program stability.

[0062] The `parse_case` function retrieves test case configuration file information, including the test data name, test template path, and test data configuration file path. After retrieving the information, it calls the `parse_toml` function.

[0063] `parse_toml` parses the TOML file and validates the test data parsed by `parse_case`. If there are no problems, it calls the function `mpl_exec` to create test cases based on the parsed parameters, test template, and the path to the test cases.

[0064] mpl_exec is used to create test cases using multiple threads. The function body calls parse_pyrun to parse the pyRun.sh file, copy the test template, and parse the common parts of multiple test cases to be assembled, and finally pass them to make_case.

[0065] `make_case` generates unit test cases by replacing variables in the copied template with test data, calls the `gen_run_cmd` function to source the preprocessed file, prepares the environment for running the unit test script, runs the script to generate a test case log file, parses the file, and checks whether the verification result of the verification function matches the expected result. If incorrect, it returns an error. If correct, it packages the test cases into a compressed package.

[0066] After the test cases are created, the function parse_toml will run the script pyRun_reg.sh to check the correctness of all test case archives. The function of pyRun_reg.sh is to decompress the archives, prepare the environment, run the test cases, check the correctness of the files, and count the number of failed test cases.

[0067] Example 2

[0068] In this embodiment, an electronic device is also provided, comprising a processor and a memory. The memory is used to store non-transitory computer-readable instructions (e.g., one or more computer program modules). The processor is used to execute the non-transitory computer-readable instructions, which, when executed by the processor, can perform one or more steps of the batch test case generation and testing method described above.

[0069] For example, the processor can be a central processing unit (CPU), a digital signal processor (DSP), or other processing units with data processing and / or program execution capabilities, such as a field-programmable gate array (FPGA); for example, the central processing unit (CPU) can be an x86 or ARM architecture.

[0070] For example, the memory may include any combination of one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD and ROM), USB storage, flash memory, etc. One or more computer program modules may be stored on the computer-readable storage medium, and the processor may run one or more computer program modules to implement various functions of the electronic device. Various application programs and various data, as well as various data used and / or generated by the application programs, may also be stored in the computer-readable storage medium.

[0071] It should be noted that, in the embodiments of this application, the specific functions and technical effects of the electronic devices can be referred to the description above of a method for generating and testing batch test cases, and will not be repeated here.

[0072] Example 3

[0073] In this embodiment, a computer-readable storage medium is also provided for storing non-transitory computer-readable instructions. For example, when the non-transitory computer-readable instructions are executed by a computer, one or more steps in the batch test case generation and testing method described above can be performed.

[0074] For example, this storage medium can be used in the aforementioned electronic device. For example, the storage medium can be the memory in the electronic device of Embodiment 2. For example, relevant descriptions of the storage medium can be found in the corresponding description of the memory in the electronic device of Embodiment 2, and will not be repeated here.

[0075] It should be noted that the storage medium (computer-readable medium) mentioned above in this application may be a computer-readable signal medium or a non-transitory computer-readable storage medium, or any combination of the two. A non-transitory computer-readable storage medium may be, for example,, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a non-transitory computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0076] In this application, a non-transitory computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can also be any computer-readable medium other than a non-transitory computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0077] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0078] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include, but are not limited to, object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server.

[0079] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0080] The units described in the embodiments of this application can be implemented in software or hardware. The names of the units are not, in some cases, limiting the scope of the unit itself.

[0081] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), etc.

[0082] The above description is merely a partial embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of disclosure in this application is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this application.

[0083] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in sequential order. Multitasking and parallel processing may be advantageous in certain environments. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this application. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0084] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A method for generating and testing batch test cases, characterized in that, The method includes: A pre-configured test template is provided, which includes a fixed part and a variable part. The variable part includes test input variables and expected result variables, as well as a verification function for verifying the test results and expected results of the test cases. The verification function is integrated into the unit testing framework and is used to automatically verify whether the test results and expected results of the test cases are consistent during the unit testing process. A test data configuration file is pre-configured. The test data configuration file includes a test case data set, which includes several test case data. Each test case data includes a test case name and several data elements. The data elements are divided into test data corresponding to test input variables and expected result data corresponding to expected result variables. In response to unit tests, pre-defined scripts are used to instantiate test case data into the test template to generate test cases, and the test cases are run. Based on the verification results of the verification function, the test cases are packaged. The specific steps include: Parse the test data configuration file to obtain the data for each test case; Make several copies of the test template as test cases to be assembled, and parse out the fixed part and the variable part respectively; Replace the variables in the variable part of each test case with the data elements of each test case data to form each test case; Run each test case to generate a test case log file, parse the test case log file, and determine whether the verification result of the verification function is correct. If incorrect, return; if correct, automatically package the corresponding test case and the file generated after running that conforms to the pre-configured general file name into a compressed package.

2. The method for generating and testing batch test cases according to claim 1, characterized in that, The method further includes: Check the correctness of all test case packages and count the number of failed test cases.

3. The method for generating and testing batch test cases according to claim 1, characterized in that, The method further includes: Configure a common configuration file, which is used to configure the common filename of test cases so that files that do not belong to the common filename are deleted before packaging after the test cases are run.

4. The method for generating and testing batch test cases according to claim 1, characterized in that, The method further includes: the number of data elements in the test case data is consistent with the number of variables in the variable section.

5. An electronic device, characterized in that, include: processor; A memory having stored thereon one or more computer program instructions that run on the processor; When the processor executes the computer program instructions, it performs the batch test case generation and testing method according to any one of claims 1-4.

6. A computer-readable storage medium, characterized in that, It stores computer instructions, which, when executed, perform the steps of the batch test case generation and testing method as described in any one of claims 1-4.

Images