Data-driven unit test execution use case method and system

Abstract

The application provides a data-driven unit test execution case method and system, which comprises the following steps: merging and arranging unit test project data and writing the data into a database; the project data comprises input data, output data and project configuration information of a case; extracting input data and output data of a to-be-executed case from the database and writing the data into an intermediate carrier; using a template engine to generate a general driving file without specific case data based on the project configuration information and a preset driving template, and compiling and linking the general driving file and the code to be tested to generate a single executable driving program; running the single executable driving program, dynamically reading the input data of the case from the intermediate carrier to execute the test, and collecting output information comprising actual output data. In the application, the case data is transmitted by other means during driving, and a plurality of cases only need to be driven once, which greatly improves the execution efficiency.

Description

Technical Field

[0001] This invention relates to the field of software testing technology, and more specifically, to a data-driven unit test execution method and system. Background Technology

[0002] In traditional unit testing, test case inputs, expected outputs, and other related data typically need to be converted into concrete code and reflected in the corresponding driver file. As the number of test cases increases, the amount of code containing test case data accumulates in the driver file, easily leading to a high driver code bloat rate, which in turn increases the generation, maintenance, and compilation costs of the driver file. To reduce the impact of test case data on the size of the driver code, existing technologies often employ a method of separating test case data from the driver code, thus mitigating the driver code bloat problem caused by test case data expansion.

[0003] However, in traditional unit testing, one driver typically corresponds to only one test case. When multiple test cases need to be executed, multiple driver files still need to be generated and compiled separately, resulting in a cumbersome test execution process and low overall test efficiency.

[0004] Patent document CN117785672A discloses a unit testing method, apparatus, storage medium and computer equipment, but it reduces the time spent on initializing the target module that the execution environment depends on for a single test case. The execution of multiple test cases still requires a fixed amount of time.

[0005] This invention aims to provide a data-driven unit test execution method and system. The test case data is transmitted during execution through other means, and multiple test cases only need to be executed once, which greatly improves the execution efficiency. Summary of the Invention

[0006] To address the shortcomings of existing technologies, the purpose of this invention is to provide a data-driven unit test execution method and system.

[0007] A data-driven unit test execution method according to the present invention includes: Step S1: Merge and organize the unit test project data, and write it into the database; The project data includes input data and output data for use cases, as well as project configuration information; Step S2: Extract the input and output data of the test cases to be executed from the database and write them into the intermediate carrier; Step S3: Based on the project configuration information and the preset driver template, use the template engine to generate a general driver file without specific use case data, and compile and link the general driver file and the code under test to generate a single executable driver. Step S4: Run the single executable driver to dynamically read the input data of the test case from the intermediate carrier to execute the test, and collect output information containing the actual output data.

[0008] Preferred options also include: Step S5: Parse the collected output information, including expected value verification and coverage analysis.

[0009] Preferably, the intermediate carrier includes a data file and a shared memory region.

[0010] Preferably, the process of generating a generic driver file using a template engine includes: A project configuration file is generated based on the project configuration information; the template engine is used to render a preset driver template based on the project configuration file to generate the general driver file.

[0011] Preferably, the project configuration file includes an XML file, and the template engine includes Jinja2.

[0012] Preferably, the general driver file is used as an interface for reading input and output data of test cases, an interface for parsing variable information in test cases, and an interface for calling the function under test.

[0013] Preferably, step S1 includes the following sub-steps: Step S1.1: Store the user's use case data in the interface design into the database. Variables use a unique ID as an identifier, which is composed of file, function, number and type information. Step S1.2: Write the project configuration information, including test mode, coverage criteria, unit under test, function and type information, into the configuration file; also write the additional data of the test cases to be executed, including the insertion of user code and variable symbol input, into the configuration file.

[0014] Preferably, step S2 includes writing the use case data to be executed into a data file, or using other methods including shared memory regions in an environment that does not support a data file system.

[0015] Preferably, step S2 includes the following sub-steps: Step S2.1: For compilation environments that support data file systems, write the list of test cases to be executed into the data file, including the test case name and the number of executions; Step S2.2: For each use case, write all input and output data of the use case into the use case file in the form of variable ID and variable value; Step S2.3: For compilation environments that do not support data file systems, a fixed memory address is predefined as a data buffer. Use case data is written to the specified address, and subsequent drivers access the memory region through pointers to obtain the data.

[0016] A data-driven unit test execution test case system according to the present invention includes: Module M1: Merges and organizes unit test project data and writes it to the database; The project data includes input data and output data for use cases, as well as project configuration information; Module M2: Extracts the input and output data of the test cases to be executed from the database and writes them into the intermediate carrier; Module M3: Based on the project configuration information and the preset driver template, it uses the template engine to generate a general driver file without specific use case data, and compiles and links the general driver file and the code under test to generate a single executable driver. Module M4: Runs the single executable driver, dynamically reads the input data of the test case from the intermediate carrier to execute the test, and collects output information containing the actual output data.

[0017] Preferred options also include: Module M5: Parses the collected output information, including expected value verification and coverage analysis.

[0018] Preferably, the intermediate carrier includes a data file and a shared memory region.

[0019] Preferably, the process of generating a generic driver file using a template engine includes: A project configuration file is generated based on the project configuration information; the template engine is used to render a preset driver template based on the project configuration file to generate the general driver file.

[0020] Preferably, the project configuration file includes an XML file, and the template engine includes Jinja2.

[0021] Preferably, the general driver file is used as an interface for reading input and output data of test cases, an interface for parsing variable information in test cases, and an interface for calling the function under test.

[0022] Preferably, module M1 includes the following sub-modules: Module M1.1: Stores user test case data in the interface design into the database. Variables use a unique ID as an identifier, which is composed of file, function, number and type information. Module M1.2: Writes project configuration information, including test modes, coverage standards, units under test, functions, and type information, into the configuration file; it also writes additional data, such as user code and variable symbol inputs, into the test cases to be executed.

[0023] Preferably, module M2 includes writing the test case data to be executed to a data file, or using other methods including shared memory regions in an environment that does not support a data file system.

[0024] Preferably, module M2 includes the following sub-modules: Module M2.1: For compilation environments that support data file systems, writes the list of test cases to be executed to a data file, including the test case name and the number of executions. Module M2.2: For each use case, write all input and output data of the use case into the use case file in the form of variable ID and variable value; Module M2.3: For compilation environments that do not support data file systems, a predefined fixed memory address is used as a data buffer. Test case data is written to the specified address, and subsequent drivers access the memory region through pointers to retrieve the data.

[0025] Compared with the prior art, the present invention has the following beneficial effects: 1. This invention solves the code bloat problem by centrally storing all test case data and configuration information in a database and completely separating test data from driver code, so that the code size of the general driver file does not increase with the number of test cases.

[0026] 2. This invention achieves "compile once, execute multiple times" by generating a single, general-purpose executable driver. The driver can continuously read and execute multiple test cases during runtime, avoiding the huge time overhead of repeatedly compiling the driver for each test case or batch of test cases, and significantly improving the overall efficiency of unit testing.

[0027] 3. In this invention, test case data is managed in a structured manner in the database, which makes it easy to maintain, query and reuse. At the same time, the decoupling of test logic and data makes the system architecture clearer and enhances the system's flexibility and maintainability.

[0028] 4. By supporting both data files and shared memory as intermediate carriers, this invention can flexibly adapt to standard development environments and embedded environments without a file system, thus expanding the scope of application of the technology. Attached Figure Description

[0029] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is a diagram of the driver file structure in this invention.

[0030] Figure 2 This is a flowchart illustrating the execution of data-driven use cases in an embodiment of the present invention.

[0031] Figure 3 This is a schematic diagram of a data-driven use case execution system in an embodiment of the present invention. Detailed Implementation

[0032] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0033] Example 1 like Figure 2 and Figure 3 As shown, this embodiment provides a data-driven unit test execution case method, including... Step 1: Merge and organize the unit test project data, including the input data, output data, and project configuration information of the test cases, and write them into the database.

[0034] In one embodiment, it specifically includes: Step 1.1: Store the user's use case data in the interface design into the database. Variables use a unique ID as an identifier, and their IDs are composed of information such as file, function, number, and type.

[0035] Step 1.2: Write the project configuration information, including test mode, coverage standard, unit under test, function, type, etc., into the configuration file driverconfig.xml. Also write any additional data, such as user code and variable symbol input, involved in the test cases to be executed into the configuration file driverconfig.xml to provide data for driver generation in step 3.

[0036] Step 2: Write the test case data to be executed to a file. In environments that do not support file systems, use other methods such as shared memory.

[0037] In one embodiment, it specifically includes: Step 2.1: For compilation environments that support data file systems, write the list of test cases to be executed into the VMC_ORDER.DATA file, including the test case name and the number of executions.

[0038] Step 2.2: For each use case, write all input and output data of the use case into the use case file VMC_CASE_xxx.DATA in the form of variable ID and variable value.

[0039] Step 2.3: For compilation environments that do not support file systems, a fixed memory address is predefined as a data buffer. Use case data is written to the specified address, and subsequent drivers access the memory region through pointers to obtain the data.

[0040] Step 3: As Figure 1 As shown, based on the project configuration file driverconfig.xml, the driver file is generated using the jinja2 template, and compilation and linking are completed.

[0041] In one embodiment, it specifically includes: Step 3.1: Parse the contents of the driverconfig.xml configuration file to provide data for the Jinja2 template.

[0042] Step 3.2: Use Python's Jinja2 library to implement template rendering and generate driver files. The template structure is as follows: 1. An interface for reading test case data, including file I / O operations when supporting the file system, and parsing memory information at fixed addresses in shared memory mode.

[0043] 2. Parse the interface for variable information in the test case, and complete the assignment operation of complex variables according to the unique ID composition method.

[0044] 3. Calling the function under test: The driver will implement the calling interface for all testable functions to support the execution of test cases for all functions.

[0045] 4. Stub function settings: All stub function calls are based on their function IDs in the database, and a unified interface is passed to complete the variable assignment operation within the corresponding stub function.

[0046] 5. Insert user code: For new user code data and variable symbol input data contained in the test case to be executed, complete the specific implementation in the corresponding unit under test in the driver.

[0047] 6. Output of actual values: After the function under test is executed or inside the stub function, the actual values ​​of the variables are output according to their types. The output method is the same as described in point 1, and different methods are used depending on whether the file system is supported.

[0048] Step 3.3: Compile and link the generated driver files according to the given compilation environment to generate an executable driver.

[0049] Step 4: Feed the use case data to the driver program and collect execution logs and output information.

[0050] Step 5: Complete the result parsing, including tasks such as expected value verification and coverage analysis.

[0051] In one embodiment, it specifically includes: Step 5.1: Parse the validation results of the test case execution. If the compilation environment supports a file system, the results will be stored as a result.txt file in the project's test case directory. Read the actual values ​​of all variables and validate them against the corresponding expected values ​​in the database.

[0052] Step 5.2: Parse the instrumentation output information of the test cases. If the compilation environment supports a file system, it will be stored as an ibr.txt file in the project's test case directory. This data represents whether the instrumentation inserted in the instrumentation file was executed. Based on the corresponding statement branches and other information in the database, analyze the data to complete the calculation of test case coverage.

[0053] Step 5.3: Parse the MCDC output information of the test case execution. If the compilation environment supports a file system, it will be stored in the project's test case directory as an mcdcResult.txt file. This data represents the instrumentation coverage of MCDC in the instrumentation file. Based on the relevant MCDC information in the data, calculate the MCDC coverage of the test cases.

[0054] Example 2 This embodiment details the implementation process of a data-driven unit test case execution method in a standard development environment with a complete file system.

[0055] This invention aims to separate test data from test logic to achieve a "compile once, execute multiple times" testing mode, thereby solving the problems of code bloat and low efficiency of repeated compilation caused by strong coupling between test case data and driver code in the prior art.

[0056] Step S1: The test engineer designs and manages the unit test project through a graphical user interface. Assume that the `int add(int a, int b)` function in the source file `my_math.c` needs to be tested. The engineer can create two test cases: `case_001` to verify 1 + 2 = 3, and `case_002` to verify -5 + 10 = 5. In the design interface, the engineer will define the input variables `a` and `b`, and the expected output variable is the function's return value. Specifically, for `case_001`, the input data is set to `a=1`, `b=2`, and the expected output data is `return_value=3`.

[0057] In step S2, all project-related data is structured and written to a central database. This database stores project configuration information, such as the path of the code under test, the list of functions under test, and test coverage standards. Simultaneously, test case data is recorded in detail, including the name of each test case, the values ​​of input variables, and the expected values ​​of output variables. To uniquely identify and manipulate each variable in subsequent processes, the system generates a globally unique identifier (ID) for each variable. This ID can be composed of information such as the source file where the variable resides, the function it belongs to, its function number, and its data type, thus ensuring its uniqueness. For example, the ID of the input parameter 'a' of the function 'add' can be represented as 'my_math.c::add::param_0::int'. In this way, the database achieves centralized and structured management of the previously scattered test assets.

[0058] In step S3, the system extracts data related to the current test task from the database and writes it to an intermediate carrier. In this embodiment, given that the environment supports a file system, the intermediate carrier is a series of data files.

[0059] Specifically, it includes: 1. Generate Project Data: The system queries the database to extract the macro-configuration information for this test task, including but not limited to the list of functions under test, all relevant variables and their unique IDs, and the list of required stub functions, and generates a project configuration file, such as an XML file named driverconfig.xml. This file forms the key basis for the subsequent generation of general driver files.

[0060] 2. Generate Test Case Data: The system queries the database to obtain a list of test cases to be executed and writes this information to an execution order file, such as VMC_ORDER.DATA. The content of this file can be case_001,1 and case_002,1, indicating that case_001 and case_002 are each executed once. Subsequently, the system generates an independent data file for each test case. Taking case_001 as an example, the system generates VMC_CASE_001.DATA, whose content is a series of key-value pairs of "variable ID - variable value," in the format my_math.c::add::param_0::int = 1 and my_math.c::add::param_1::int = 2. Similarly, the VMC_CASE_002.DATA generated for case_002 contains my_math.c::add::param_0::int = -5 and my_math.c::add::param_1::int = 10. In this way, all specific use case values ​​are stored in the data file, achieving complete separation from the driver code.

[0061] Step S4 involves template rendering to generate the driver and compilation. An automated script, such as a Python script, is triggered. It first parses the project data generated in the previous step, namely the driverconfig.xml file. Then, this script uses a template engine, such as Jinja2, and a pre-defined driver template file, such as driver_template.c.j2, to generate a generic driver file named driver.c. The driver template file driver_template.c.j2 contains a pre-defined logic framework for reading and parsing the DATA file, as well as an interface framework for calling the tested functions and stub functions, but it does not contain any specific test case values. Based on the information in driverconfig.xml, the template engine fills the template with function calls, variable declarations, etc., thereby generating a fully functional but data-independent driver.c. For example, the template might contain a loop to generate a switch-case structure containing all function calls based on all the tested functions listed in driverconfig.xml. It is important to emphasize that the generated driver.c file does not contain hard-coded data such as a=1, b=2. After driver.c is generated, the script calls a compiler, such as GCC, and a linker to compile and link this generic driver file with the code under test and any existing library files, ultimately generating a single executable driver, such as driver.exe. For test cases 001 and 002, and even hundreds or thousands of test cases, only this one compilation and linking process is required. This greatly improves testing efficiency and fundamentally solves the code bloat problem.

[0062] Step S5: Execute the driver and collect output. The system runs the single executable driver driver.exe generated in the previous step. After the driver starts, its internal logic first opens and reads the VMC_ORDER.DATA file to determine that case_001 and case_002 need to be executed sequentially. The program enters a loop. When processing case_001, it constructs the test case data file name VMC_CASE_001.DATA based on the test case name, and then opens and reads this file. The program parses the file content line by line. When it reads my_math.c::add::param_0::int = 1, it assigns the value 1 to the internal variable representing parameter a; when it reads my_math.c::add::param_1::int = 2, it assigns the value 2 to the internal variable representing parameter b. After the data assignment is completed, the driver calls the function under test, add(a, b). After the function completes execution, the driver captures its return value (3 in this example) and appends this actual output value, along with its variable ID, to a result data file (e.g., result.txt). The content written could be `case_001::my_math.c::add::return_value::int = 3`. The subsequent process of the loop is similar; the driver continues processing `case_002`, reads `VMC_CASE_002.DATA`, executes the test, and appends the result `case_002::my_math.c::add::return_value::int = 5` to `result.txt`.

[0063] Step S6: Result parsing. Another automated script starts, reading the result.txt file. For the result of case_001, the script parses the actual output value as 3. Simultaneously, the script queries the database based on the test case ID case_001 to obtain the expected output value for that test case, which is also 3. After comparison, the two values ​​match, and the test case is judged as "passed". The same comparison is performed for case_002. Finally, the system summarizes the verification results of all test cases and generates a detailed test report.

[0064] Example 3 This embodiment is primarily designed for embedded testing environments with limited resources and no file system support. In such environments, data interaction via files is no longer applicable; therefore, this embodiment uses shared memory as an intermediate medium.

[0065] The overall process of this embodiment is similar to that of embodiment 2. The main difference between this embodiment and embodiment 2 lies in step S3 and subsequent steps.

[0066] In step S3, after the system retrieves the test case data to be executed from the database, it no longer generates a DATA file. The control program on the test host requests a shared memory region. The starting address and size of this shared memory region are predefined known values ​​for both the test host and the target board under test. The control program serializes the execution order information of the test cases (equivalent to the content of VMC_ORDER.DATA) and the specific input and output data of each test case (equivalent to the content of VMC_CASE_xxx.DATA) according to a predefined memory layout structure, and then writes them to the shared memory region all at once or in batches. For example, the beginning of the memory region can be set as a header to describe the number of test cases and the offset address and length of each test case data block, followed by the data blocks of each test case.

[0067] In step S4, the template no longer contains file I / O operation functions such as fopen and fread, but instead provides logic for directly accessing and parsing memory data through pointers. For example, the driver code might include a pointer to a predefined shared memory starting address and read use case data through pointer offset and type conversion. The generated generic driver file driver.c is compiled and linked using a cross-compiler suitable for the target embedded system to generate an executable file that can run on the target board.

[0068] In step S5, the executable driver is downloaded to the target board and run. After startup, the driver directly accesses the specified shared memory address via a pointer, first reading the header information to understand the overall situation of the test task, then parsing the data blocks of each test case sequentially, assigning values ​​to test variables, and calling the function under test. The actual output data and log information generated after test execution are no longer written to a file, but are written back to another pre-specified result area in shared memory.

[0069] In step S6, the monitoring program on the test host determines whether the test is complete by polling or waiting for signals from the target board. Once it detects that the shared memory result area has been updated, the monitoring program reads the data from it. Since the data format is predefined, the monitoring program can accurately parse the actual output value of each test case. The subsequent processes, such as comparing with the expected values ​​in the database and generating test reports, are completely consistent with Example 2.

[0070] The technical solution of this embodiment can be effectively applied to special environments such as embedded systems and real-time operating systems without a file system, which greatly expands the scope of application of the technology of this application and demonstrates the flexibility and environmental adaptability of the solution.

[0071] Example 4 Generic driver files are generated by a template engine (such as Jinja2) based on project configuration files (such as driverconfig.xml) and driver templates (such as driver_template.c.j2). Their modular design is the key to achieving "genericity" and "flexibility".

[0072] like Figure 1 As shown, a typical general-purpose driver file can be logically divided into multiple functional modules.

[0073] The data interaction module enables communication between the driver and external data sources. This module's implementation is configurable. During the template rendering phase, the template engine can generate different code based on the passed context variables (e.g., `use_filesystem`). In the scenario of Example 2, this module will contain code that uses C standard library functions (such as `fopen`, `fgets`, and `sscanf`) to open and read the DATA file line by line. In the scenario of Example 3, this module contains code that directly accesses predefined shared memory addresses using pointers and obtains the original data blocks through `memcpy` and pointer arithmetic. This design allows the driver to seamlessly adapt to different runtime environments.

[0074] The variable assignment module is responsible for parsing the raw data obtained from the data interaction module and assigning it to the corresponding test variables in the program. The core of this module is typically a large switch-case structure or an array of function pointers, with dispatch based on the unique ID of the variable. When an "ID-value" pair is read from the data source, the program enters the corresponding processing branch based on the ID. For example, for a variable with the ID my_math.c::add::param_0::int, the code in the case branch can execute operations like sscanf(value_string, "%d",&g_add_param_a); or g_add_param_a = *(int*)data_ptr;. For complex structure or array variables, the assignment logic is more complex, potentially involving recursive calls or loop assignments, but this complex logic can also be automatically generated by the template based on the type information defined in driverconfig.xml, thus providing a unified variable parsing capability.

[0075] The module for calling the function under test (DUT) is the core module for executing tests. It is typically implemented using a dispatch structure (such as a switch-case statement), which calls the corresponding DUT based on the function ID required for the current test case. During rendering, the template engine iterates through all DUTs declared in driverconfig.xml, generating a call entry point for each function. For example, `case FUNC_ID_ADD: result = add(g_add_param_a, g_add_param_b); break;` This ensures that a single driver program has the ability to call any DUT in the project, forming a DUT call interface.

[0076] The stub function module is used in unit testing to simulate the behavior of other functions or modules that the unit under test depends on. This module provides a unified mechanism for setting and calling stub functions. When the code under test calls a function that has been stubbed, the execution flow is redirected to a unified stub function processing entry point. This entry function receives the ID of the called stub function as a parameter and internally simulates the behavior of the stub function based on this ID and preset behaviors read from the test case data (e.g., assigning a specific value to a certain output parameter of the stub function, or returning a specific value). In one embodiment, if the test case data contains `STUB_FUNC_GET_SENSOR_DATA::return_value = 100`, the stub function module will ensure that the call to `GetSensorData()` returns 100 during execution. This module implements a stub function setting interface.

[0077] The user code insertion module is designed to provide greater flexibility. Specific "hooks" or placeholders are reserved in the template, such as before and after calling the function under test. Test engineers can write small C code snippets for specific test cases and store them in the database when designing test cases. When generating the driver, these code snippets are read from driverconfig.xml and precisely inserted into the corresponding positions in the driver file by the template engine.

[0078] The actual value output module is responsible for collecting output information containing the actual output values ​​after the function call module under test has finished executing. In a file system environment, it uses fprintf to write a string in the format "test case ID::variable ID = actual value" to a result file (such as result.txt). In a file system-less environment, it uses memcpy to write the actual values ​​to the shared memory result area.

[0079] These modules work together to ensure that the final driver file is powerful and flexible while maintaining universality, capable of handling various complex testing scenarios. The complexity is handled by the templates and generation tools, while maintaining a simple interface for users.

[0080] Example 5 This embodiment focuses on the result parsing stage. The result parsing process is usually executed by one or more post-processing scripts, which run automatically after all test cases have been executed.

[0081] The first step is correctness verification, which aims to confirm whether the functional logic of the code under test meets expectations. The parsing script first reads the raw result data generated by the actual value output module, which in Example 2 is part of the result file, such as the result.txt file. This file contains the actual output values ​​of all executed test cases, in the format described above, for example, case_001::my_math.c::add::return_value::int = 3. For each record, the script parses out the test case ID (case_001), variable ID (my_math.c::add::return_value::int), and actual value 3. Subsequently, the script uses the test case ID and variable ID as indexes to query the database to obtain the expected output value set when designing the test case. After obtaining the expected value 3, the script performs a comparison operation. For integer and string data, exact equality comparisons are typically performed. However, for floating-point numbers, to avoid misjudgments due to precision issues, a small allowable error range is introduced during comparison. For example, if abs(actual_value - expected_value) < 0.0001, the two are considered equal. If the actual value and the expected value are within the allowable error range, the validation passes; otherwise, it fails. The script records all failed validation items, including the test case name, variable name, expected value, and actual value, and finally summarizes them in the test report.

[0082] The second step is code coverage analysis. This analysis assumes that the source files of the code under test (such as my_math.c) have been instrumented before the compilation and linking in step S4. The instrumentation process is performed by a specialized tool that analyzes the source code structure and inserts probe code at key locations. For example, to calculate statement coverage, probes are inserted at the entry point of each basic code block; to calculate branch coverage, probes are inserted at the true / false branches of each conditional statement such as if and while. All these probe points are assigned a unique ID, and their information is recorded in a database.

[0083] During the execution of the driver in step S5, whenever the execution flow passes through an instrumentation point, the corresponding probe code is triggered, and the ID of the probe is recorded to an output file, such as ibr.txt, or written to a specific area of ​​shared memory.

[0084] In the result parsing phase of step S6, the coverage analysis script reads the ibr.txt file (or the corresponding memory area) to obtain a set of all probe IDs that were triggered in this test. Then, the script retrieves the total set of probe IDs for the project from the database. The code coverage is obtained by calculating the ratio of the "number of triggered probe IDs" to the "total number of probe IDs". For example, statement coverage = (number of executed basic blocks / total number of basic blocks) * 100%. Branch coverage is calculated similarly.

[0085] This application also supports modified decision condition coverage MCDC analysis. MCDC requires demonstrating in testing that each independent condition in a complex decision expression independently affects the final decision result. This necessitates more sophisticated instrumentation techniques and result recording methods. During execution, a dedicated MCDC result file, such as mcdcResult.txt, is generated. This file details the combination of values ​​for each independent condition within each decision expression during execution, as well as the final decision result. The MCDC analysis script parses this file and, based on the definition of MCDC, determines whether the MCDC coverage target for each decision expression has been achieved, ultimately calculating the MCDC coverage rate for the entire project.

[0086] Through the detailed correctness verification and various code coverage analyses, including statement coverage, branch coverage, and MCDC coverage, this embodiment can determine whether the function is correct and can also evaluate the depth and breadth of testing with quantitative data, providing a reliable guarantee for software quality.

[0087] This invention also provides a data-driven unit test execution test case system, comprising: Module M1: Merges and organizes unit test project data and writes it to the database; The project data includes input data and output data for use cases, as well as project configuration information; Module M2: Extracts the input and output data of the test cases to be executed from the database and writes them into the intermediate carrier; Module M3: Based on the project configuration information and the preset driver template, it uses the template engine to generate a general driver file without specific use case data, and compiles and links the general driver file and the code under test to generate a single executable driver. Module M4: Runs the single executable driver, dynamically reads the input data of the test case from the intermediate carrier to execute the test, and collects output information containing the actual output data.

[0088] Also includes: Module M5: Parses the collected output information, including expected value verification and coverage analysis.

[0089] The intermediate carrier includes data files and a shared memory area.

[0090] The process of generating generic driver files using a template engine includes: A project configuration file is generated based on the project configuration information; the template engine is used to render a preset driver template based on the project configuration file to generate the general driver file.

[0091] The project configuration file includes an XML file, and the template engine includes Jinja2.

[0092] The general driver file is used as an interface for reading input and output data of test cases, an interface for parsing variable information in test cases, and an interface for calling the function under test.

[0093] The module M1 includes the following sub-modules: Module M1.1: Stores user test case data in the interface design into the database. Variables use a unique ID as an identifier, which is composed of file, function, number and type information. Module M1.2: Writes project configuration information, including test modes, coverage standards, units under test, functions, and type information, into the configuration file; it also writes additional data, such as user code and variable symbol inputs, into the test cases to be executed.

[0094] The module M2 includes writing the test case data to be executed to a data file, and using other methods including shared memory regions in environments that do not support data file systems.

[0095] Module M2 includes the following sub-modules: Module M2.1: For compilation environments that support data file systems, writes the list of test cases to be executed to a data file, including the test case name and the number of executions. Module M2.2: For each use case, write all input and output data of the use case into the use case file in the form of variable ID and variable value; Module M2.3: For compilation environments that do not support data file systems, a predefined fixed memory address is used as a data buffer. Test case data is written to the specified address, and subsequent drivers access the memory region through pointers to retrieve the data.

[0096] Those skilled in the art will understand that, besides implementing the system and its various devices, modules, and units provided by this invention in the form of purely computer-readable program code, the same functions can be achieved entirely through logical programming of the method steps, enabling the system and its various devices, modules, and units to function in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, the system and its various devices, modules, and units provided by this invention can be considered a hardware component, and the devices, modules, and units included therein for implementing various functions can also be considered structures within the hardware component; alternatively, the devices, modules, and units for implementing various functions can be considered both software modules implementing the method and structures within the hardware component.

[0097] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A data-driven unit test execution method, characterized in that, include: Step S1: Merge and organize the unit test project data, and write it into the database; The project data includes input data and output data for use cases, as well as project configuration information; Step S2: Extract the input and output data of the test cases to be executed from the database and write them into the intermediate carrier; Step S3: Based on the project configuration information and the preset driver template, use the template engine to generate a general driver file without specific use case data, and compile and link the general driver file and the code under test to generate a single executable driver. Step S4: Run the single executable driver to dynamically read the input data of the test case from the intermediate carrier to execute the test, and collect output information containing the actual output data.

2. The data-driven unit test execution method according to claim 1, characterized in that, Also includes: Step S5: Parse the collected output information, including expected value verification and coverage analysis.

3. The data-driven unit test execution method according to claim 1, characterized in that, The intermediate carrier includes data files and a shared memory area.

4. The data-driven unit test execution method according to claim 1, characterized in that, The process of generating generic driver files using a template engine includes: A project configuration file is generated based on the project configuration information; the template engine is used to render a preset driver template based on the project configuration file to generate the general driver file.

5. The data-driven unit test execution method according to claim 4, characterized in that, The project configuration file includes an XML file, and the template engine includes Jinja2.

6. The data-driven unit test execution method according to claim 1, characterized in that, The general driver file is used as an interface for reading input and output data of test cases, an interface for parsing variable information in test cases, and an interface for calling the function under test.

7. The data-driven unit test execution method according to claim 1, characterized in that, Step S1 includes the following sub-steps: Step S1.1: Store the user's use case data in the interface design into the database. Variables use a unique ID as an identifier, which is composed of file, function, number and type information. Step S1.2: Write the project configuration information, including test mode, coverage criteria, unit under test, function and type information, into the configuration file; also write the additional data of the test cases to be executed, including the insertion of user code and variable symbol input, into the configuration file.

8. The data-driven unit test execution method according to claim 3, characterized in that, Step S2 includes writing the test case data to be executed into a data file, or using other methods including shared memory regions in an environment that does not support a data file system.

9. The data-driven unit test execution method according to claim 8, characterized in that, Step S2 includes the following sub-steps: Step S2.1: For compilation environments that support data file systems, write the list of test cases to be executed into the data file, including the test case name and the number of executions; Step S2.2: For each use case, write all input and output data of the use case into the use case file in the form of variable ID and variable value; Step S2.3: For compilation environments that do not support data file systems, a fixed memory address is predefined as a data buffer. Use case data is written to the specified address, and subsequent drivers access the memory region through pointers to obtain the data.

10. A data-driven unit test execution test case system, characterized in that, include: Module M1: Merges and organizes unit test project data and writes it to the database; The project data includes input data and output data for use cases, as well as project configuration information; Module M2: Extracts the input and output data of the test cases to be executed from the database and writes them into the intermediate carrier; Module M3: Based on the project configuration information and the preset driver template, it uses the template engine to generate a general driver file without specific use case data, and compiles and links the general driver file and the code under test to generate a single executable driver. Module M4: Runs the single executable driver, dynamically reads the input data of the test case from the intermediate carrier to execute the test, and collects output information containing the actual output data.

Images