An automated testing framework and method

Abstract

The present invention provides an automated testing framework and method for performing software testing on offline trackside safety platform tools based on pre-written test case documents. The automated testing framework comprises: a tool-to-be-tested module; a standard input module; a use case module, including a use case input submodule and a use case output submodule; a configuration module; and a test module. The test module automatically generates a use case input submodule based on the configuration module and the standard input module. This invention addresses technical issues encountered during the validation testing process of offline trackside safety platform tools in rail transit signaling systems, significantly improving efficiency compared to manual testing methods and offering high reusability.

Description

Technical Field

[0001] The present invention relates to the fields of rail transit signals and software testing, and in particular to an automated testing framework and method for offline tools of a trackside safety platform. Background Art

[0002] The safety computer trackside platform can be used in signal systems with a safety level defined as SIL4 (Safety Integrity Level-4) in subways, national railways, or rail transit systems, such as tram switch controllers, national railway train control and interlocking integrated systems / radio block centers / temporary speed limit servers, urban rail intelligent safety computer interlocking systems / area controllers / data storage units, etc. When the safety computer trackside platform is loaded with SIL4 safety applications to form an overall SIL4-level system product, the safety of the entire system can meet SIL4 requirements. The offline tool used with the trackside safety platform must comply with the development requirements of EN50128-2011 for SIL2 software. To ensure the stable operation of the offline tool and generate accurate configuration data, the offline tool must be fully tested before release to ensure that the tool can meet various scenarios and requirements. Currently, the testing of the trackside safety platform offline tool is entirely done manually, which has the following main disadvantages:

[0003] 1. The number of test cases is large, and any data changes in the input files must be manually rewritten and the MD5 values updated simultaneously. When the changed data involves multiple station numbers, folders must be switched back and forth to find the input files for the corresponding stations. After the input files involved in each step are changed, the offline tool must be manually run by clicking the executable file, which is time-consuming and labor-intensive, with numerous repetitive steps and low testing efficiency.

[0004] 2. The test success or failure indicators are completely manually checked, resulting in a certain defect escape rate;

[0005] 3. The test records and result statistics in the test report need to be recorded manually one by one, and a detailed test report cannot be automatically output;

[0006] 4. Poor reusability. When multiple rounds of testing are involved, regression cannot be achieved with one click. It is necessary to repeatedly copy and paste relevant files in previous test station numbers as input and re-execute offline tools, involving a lot of repetitive work. Summary of the Invention

[0007] The purpose of the present invention is to provide an automated testing framework and method to solve the technical problems existing in the confirmation test process of the offline tool of the trackside safety platform in the rail transit signal system. Compared with the manual testing method, it greatly improves the efficiency and has strong reusability.

[0008] To achieve the above object, the present invention is implemented through the following technical solutions:

[0009] An automated testing framework is provided for performing software testing on an offline tool of a trackside safety platform based on pre-written test case documents, wherein the test case documents include multiple use cases, and the software test is composed of tests of each use case. The automated testing framework includes:

[0010] The tool module to be tested is the offline tool of the trackside safety platform;

[0011] The standard input module is a collection of all standard input files that can be used for the successful operation of the tool module under test;

[0012] The use case module includes a use case input submodule and a use case output submodule; wherein the use case input submodule is a collection of input files required to execute each use case, which is manually created or automatically generated based on the standard input module; the use case output submodule is a collection of output files generated after executing each use case;

[0013] Configuration module, used to set and save parameter configuration of the automated testing framework;

[0014] The test module is used to perform automated testing on the tool module to be tested based on the use case input submodule and configuration module of the use case module, and generate a use case output submodule; wherein, the test module can automatically generate a use case input submodule based on the configuration module and the standard input module.

[0015] Preferably, each use case includes one or more use case steps, and each use case test consists of tests of each use case step.

[0016] The use case input submodule includes a plurality of use case step input file sets, each of which is a collection of one or more use case step input files required to execute the use case step;

[0017] The test module includes a unit for automatically generating a use case step input file set, which can automatically generate a use case step input file set based on a configuration module and a standard input module. Each use case step input file set constitutes a use case input submodule.

[0018] Preferably, the configuration module includes a basic information configuration unit and a use case configuration unit; wherein,

[0019] The basic information configuration unit includes an automatic generation of a use case input submodule identifier, which is used to indicate whether to automatically generate the use case input submodule;

[0020] The use case configuration unit includes a plurality of use case step configuration subunits for configuring relevant parameters for executing each use case step, each of the use case step configuration subunits corresponding to each use case step; each of the use case step configuration subunits includes automatic generation parameters for automatically generating a use case step input file set;

[0021] The automatic generation of use case step input file set unit automatically generates the use case step input file set based on the automatic generation of use case input submodule identifier and automatic generation parameters.

[0022] Preferably, each use case step input file of the use case step input file set may be an input file prepared manually in accordance with the use case step, or may be a standard input file copied and rewritten from the standard input module.

[0023] Preferably, the automatically generated parameters include the copy file name, the rewrite field name and the rewrite target value, which are respectively used to indicate the name of the standard input file that needs to be copied and rewritten, the field name that needs to be rewritten in the standard input file, and the rewrite target value of the field.

[0024] Preferably, whether the automatic use case generation step input file set unit is executed or not is determined based on the read automatic use case generation input submodule identifier;

[0025] When the automatic generation of use case step input file set unit is running, it automatically establishes a use case input submodule directory and the corresponding use case step input file set directory under it based on the automatic generation parameters of the current use case step, and then copies the required standard input files in the standard input module to the established use case step input file set directory and rewrites it to form the corresponding use case step input file set.

[0026] Preferably, the test module includes an execution unit for calling the tool module to be tested to execute each use case step based on the input file set of each use case step and outputting the execution result.

[0027] Preferably, the use case output submodule includes a plurality of use case step output file sets, each of which is a collection of one or more use case step output files output by executing the use case step;

[0028] The output execution result of the execution unit includes establishing a use case output submodule directory and a corresponding use case step output file set directory thereunder, and placing the execution result into the use case step output file set directory.

[0029] Preferably, each of the use case step configuration subunits includes:

[0030] Use case-step number, used to indicate the use case step corresponding to the configuration subunit of this use case step;

[0031] Execution flag, used to indicate whether this use case step is executed;

[0032] The execution unit determines whether to execute the use case step corresponding to the use case step number based on the execution identifier.

[0033] Preferably, each of the use case step configuration subunits includes result interpretation parameters, and the result interpretation parameters include expected results, which are manually set expected execution results after executing the use case step and are used as a basis for interpreting the execution results;

[0034] The test module includes a judgment unit for judging the expected result of the parameter based on the result judgment, comparing it with the execution result in the corresponding output file set of the use case step, and automatically checking whether the execution result is correct.

[0035] Preferably, the judgment unit compares the execution result and the result judgment parameter by an assertion method.

[0036] Preferably, the execution result of the execution unit includes log information, or generates data and log information;

[0037] The result interpretation parameters include expected data values and expected log information; wherein, if the execution result is that the generated data meets expectations, the expected data value is set, which is the expected generated data value of each result field in the output file of each use case step after executing the use case step; if the execution result is that the running ends successfully or the error exit meets expectations, the expected log information is set, which is the log expected to be generated after executing the use case step;

[0038] The inspection of the execution result by the judgment unit is a log information inspection or a generated data inspection, wherein the log information inspection is performed when the expected result of the use case step is a successful completion of the operation or an error exit, and the log information in the execution result is compared with the expected log information in the result judgment parameter; the generated data inspection is performed when the expected result of the use case step is that the generated data meets expectations, and the generated data in the execution result is compared with the expected data value in the result judgment parameter.

[0039] Preferably, the result interpretation parameter includes a test success / failure flag, which is used to indicate whether the test of the use case step is passed, and the interpretation unit automatically fills in the flag according to the interpretation execution result.

[0040] Preferably, the expected data value may be an exact value, a range, an enumeration or a regular expression.

[0041] Preferably, the test module includes a reporting unit for collecting test results and outputting a test report after completing the current round of testing.

[0042] Preferably, the test report may include part or all of the details of the total number of use cases, the number of executed use cases, the use case execution rate, the test case pass rate, the number of passed use cases, and the number of failed use cases.

[0043] Preferably, the test module includes a regression test unit for performing one-click regression testing.

[0044] Preferably, the regression test can choose to use the use case input submodule of the previous round of testing for one-click regression testing, or it can perform one-click regression testing by automatically rebuilding the use case input submodule by modifying the automatic generation of the use case input submodule identifier and automatic generation parameters of the configuration module.

[0045] Preferably, the basic information configuration unit further includes module path information for setting the root path information of each module.

[0046] Preferably, the test module further includes a configuration reading unit, which is used to read and store configuration information in the configuration module and provide it to other units of the test module for use.

[0047] An automated testing method, implemented based on any of the above automated testing frameworks, comprises the following steps:

[0048] S1. Build an automated testing framework that includes at least a tool module to be tested, a standard input module, a configuration module, and a test module;

[0049] S2. Run the test module and read and store all configuration information of the configuration module by reading the configuration unit;

[0050] S3. Based on the read configuration information, traverse each expected use case step and test it one by one, and automatically interpret the test results.

[0051] Preferably, step S1 includes:

[0052] Import the tool module to be tested and the standard input module; establish a test module; establish a configuration module and configure its parameters; establish the root directory of the use case module.

[0053] Preferably, if the current round of testing requires manual creation of a use case input submodule, step S1 further includes:

[0054] Manually create a use case input submodule directory and each use case step input file set directory under the use case module root directory, and then manually create each use case step input file and place it in each corresponding use case step input file set directory to form a use case step input file set. Each use case step input file set constitutes a use case input submodule.

[0055] Preferably, step S3 includes:

[0056] S31. Based on the read configuration information, determine whether to execute the automatic generation of the use case step input file set unit to automatically establish the use case step input file set;

[0057] S32. Based on the input file set of the use case step, the execution unit calls the tool module to be tested to execute the current use case step and outputs the execution result;

[0058] S33. Based on the read configuration information, the unit test execution result is judged, and a test success / failure flag is assigned according to the test result.

[0059] Preferably, step S3 further includes step S4: a reporting unit collects statistics on the test results of step S3 and outputs a test report.

[0060] Preferably, step S4 further includes step S5: when regression testing is required, performing one-key regression testing through the regression testing unit.

[0061] In summary, compared with the prior art, the automated testing framework and method provided by the present invention have the following beneficial effects:

[0062] 1. With the automated testing framework, testers no longer need to modify input data step by step. Instead, the test tool automatically completes the task, reducing manual modification time, greatly improving testing efficiency, and reducing the burden on testers, allowing them to focus on ensuring the accuracy of test case files.

[0063] 2. Test results are visualized. The test framework automatically determines the execution results. After the test is completed, the test pass status will be displayed, and a test report will be generated as required, which includes the total number of test cases, the number of test cases executed this time, the test case execution rate, the test case pass rate, and the details of the passed and failed test case numbers.

[0064] 3. Strong reusability. Testers can decide whether to conduct regression testing based on whether there are any failed use cases after a round of execution. By updating the configuration file, regression testing can be performed on a specified range of use case sets. During the retesting process, the existing input files of the previous round can be used or the standard input files can be rewritten and executed. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 is a flow chart of the automated testing method of the present invention;

[0066] Figure 2 FIG. 1 is a schematic diagram of an embodiment of the automated testing framework of the present invention. DETAILED DESCRIPTION

[0067] The following is a further detailed description of an automated testing framework and method proposed in the present invention in conjunction with the accompanying drawings and specific implementation methods. According to the following description, the advantages and features of the present invention will be clearer. It should be noted that the drawings are in a very simplified form and use non-precise proportions. They are only used to conveniently and clearly assist in explaining the purpose of the implementation of the present invention, and are not used to limit the limiting conditions for the implementation of the present invention. Therefore, they have no technical significance. Any structural modification, change in proportional relationship or adjustment of size should still fall within the scope of the technical content disclosed by the present invention without affecting the efficacy and purpose that can be achieved by the present invention.

[0068] It should be noted that, in the present invention, relational terms such as and are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "include," "comprise," or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or apparatus comprising a series of elements includes not only the elements explicitly listed, but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus.

[0069] Combined with attachment Figures 1-2 The present invention provides an automated testing framework and method for performing software testing on an offline tool of a trackside safety platform based on a pre-written test case document, wherein the test case document includes multiple use cases, each use case includes one or more use case steps, and the software test is composed of tests of each use case step. The automated testing framework includes a tool module to be tested, a standard input module, a use case module, a configuration module, and a test module; wherein,

[0070] The tool module to be tested is an offline tool for the trackside safety platform. Figure 2 As shown, it is the "OfflineTool" folder in the root directory of the test framework.

[0071] The standard input module is a collection of all standard input files required for the tool module under test to successfully run. In this embodiment, it is the "StandardInput" folder in the root directory of the test framework. It contains standard input files such as device information and communication protocol parameters for local and remote devices at each station, configuration information for various protocols, and matching MD5 files.

[0072] The use case module includes a use case input submodule and a use case output submodule; wherein, the use case input submodule is manually established or automatically generated based on the standard input module, and is a collection of input files required to execute each use case, which includes multiple use case step input file sets, each use case step input file set is a collection of one or more use case step input files required to execute the use case step, each use case step input file may be an input file manually prepared to comply with the use case step, or may be a standard input file copied and rewritten from the standard input module; the use case output submodule is a collection of output files generated after executing each use case, including multiple use case step output file sets, each use case step output file set is a collection of one or more use case step output files output by executing the use case step. In this embodiment, the "TestCase" folder under the root directory of the test framework is the use case module, and the "Input" folder therein is the use case input sub-module, and the folders therein named with "use case number-step number" are the input file sets of each use case step, which store the input files of each use case step required to execute the use case step (such as the "TestCase1-step1" folder stores the input files required to execute step 1 of use case No. 1); the "Output" folder therein is the use case output sub-module, and the folders therein named with "use case number-step number" are the output file sets of each use case step, which store the output files of each use case step output by executing the use case step.

[0073] The configuration module is used to set and save the parameter configuration of the automated test framework; in this embodiment, it is the "testConfig.xlsx" file in the "Configuration" folder in the root directory of the test framework; it includes a basic information configuration unit and a use case configuration unit, where:

[0074] The basic information configuration unit includes module path information and an automatically generated use case input submodule identifier; wherein, the module path information is the root path information of each module; the automatically generated use case input submodule identifier is used to indicate whether to automatically generate a use case input submodule, including two options: "Yes" and "No".

[0075] The use case configuration unit includes multiple use case step configuration subunits corresponding to each use case step, which are used to configure the relevant parameters for executing each use case step. Each use case step configuration subunit includes:

[0076] Use case-step number, indicating the use case step corresponding to the configuration subunit of this use case step;

[0077] Execution flag, indicating whether the use case step is executed, including two options: "Yes" and "No";

[0078] Automatically generated parameters are related parameters of the use case step input file set automatically generated by this use case step, including the copy file name, rewrite field name, and rewrite target value, which are used to indicate the name of the standard input file that needs to be copied and rewritten in this use case step, the name of the field that needs to be rewritten in the standard input file, and the rewrite target value of the field;

[0079] The result interpretation parameters include expected results and test success / failure indicators. The expected result is the expected execution result after executing the use case step manually set, which is used as the basis for interpreting the execution result, and includes expected data values and expected log information. If the execution result is that the generated data meets expectations, the expected data value is set, which is the expected generated data value of each result field in the output file of each use case step after executing the use case step. The expected log information is the log expected to be generated after executing the use case step, which includes two types: expected success log and expected error log. If the execution result is that the successful operation ends in accordance with expectations, the expected log information is the expected success log; if the execution result is that the error exit meets expectations, the expected log information is the expected error log, which contains the expected error statement. The test success / failure indicator is used to indicate whether the test of the use case step passes, and includes two options: "Pass" and "Fail", which are automatically filled in according to the interpretation of the execution result. In some embodiments, the expected data value does not need to be configured with an exact value, and can be in the form of a range, enumeration, regular expression, etc.

[0080] The test module is used to automatically detect the tool module under test based on the standard input module, use case module and configuration module; in this embodiment, it is the "Test.bat" file in the root directory of the test framework. The test module includes:

[0081] The configuration reading unit is used to read and store the configuration information in the configuration module and provide it to other units of the test module for use;

[0082] The automatic generation of use case step input file set unit is used to automatically generate the use case step input file set of the use case step based on the automatic generation parameters in the read configuration information. Whether this unit is run or not is determined based on the read automatic generation use case input submodule identifier. If the automatic generation of use case input submodule identifier is "yes", this unit is executed. Based on the automatic generation parameters of the current use case step read, it automatically establishes a use case input submodule directory (if it already exists, it will not be repeatedly established) and the corresponding use case step input file set directory under it, and then copies and rewrites the required standard input files in the standard input module to the established use case step input file set directory to form the use case step input file set; if the automatic generation of use case input submodule identifier is "no", this unit is not executed, and it is necessary to manually establish the use case input submodule, that is, manually establish the use case input submodule directory and the use case step input file set directories under it, and then manually prepare the use case step input files that meet the use case step and place them in the corresponding use case step input file set directories, finally forming a use case input submodule composed of the use case step input file sets. The manual method is usually suitable for situations where automatic rewriting is not convenient.

[0083] The execution unit is used to call the tool module to be tested to execute each use case step based on the input file set of each use case step, and output the execution result. The execution unit takes the corresponding use case step input file set as input, runs the tool module to be tested to generate the execution result, establishes the use case output submodule directory (if it already exists, it will not be re-established) and the corresponding use case step output file set directory under it, and puts the execution result into the use case step output file set directory. The execution result includes log information and generated data. The log information includes a successful operation log and an error log. When the use case step runs successfully, the output log information is a successful operation log, and its execution result includes generated data in addition to the log information; otherwise, the output log information is an error log containing an error statement, and the error statement is used to indicate a specific operation error.

[0084] The judgment unit verifies the execution result output by the execution unit based on the read configuration information, and assigns a test success or failure flag of the configuration information according to the verification result. The judgment unit compares the result judgment parameter and the execution result in the corresponding use case step output file set through the assertion method; the verification of the execution result is log information verification or generated data verification, wherein the log information verification includes two situations: when the expected result of the use case step is to successfully run to the end, the specific value of the generated data is not concerned at this time (for example, whether the boundary value of a data in the input file of a certain use case step can make the tool module under test run successfully to the end, at this time, the specific value of the generated data is not concerned, and only the boundary value of the input file is expected to run smoothly to the end), so it is only necessary to judge whether the output successful operation log meets the expected successful log in the configuration information; when the expected result of the use case step is to exit with an error, no data is generated in this case, then it is judged whether the error statement in the error log of the output execution result meets the expected error log in the configuration information. Generated data verification applies to use case steps where the expected result is generated data that meets expectations. Generating data requires that the use case step successfully completes (because if an error occurs, no output will be generated). Therefore, in this case, it is only necessary to determine whether the generated data meets the expected results in the configuration module, without verifying the log information. Finally, if the comparison results are completely consistent, the test success / failure indicator is set to "Pass", otherwise it is set to "Fail".

[0085] The reporting unit is used to compile statistics on the test results and output a test report after the current round of testing is completed (all use case steps marked "yes" have been executed). The test report can include the total number of use cases, the number of executed use cases, the use case execution rate, the test case pass rate, and a detailed list of the number of passed and failed use cases, etc., which can help testers decide whether to conduct regression testing.

[0086] The regression test unit is used to perform one-click regression testing. Whether to perform regression testing is determined by the tester based on the test report output by the reporting unit. Regression testing can be performed using the use case input submodule from the previous test, or by automatically rebuilding the use case input submodule by modifying the auto-generate use case input submodule identifier and auto-generation parameters in the configuration module.

[0087] In addition, as attached Figure 1 As shown, the automated testing method includes the steps of:

[0088] S1. Build an automated testing framework. This includes: importing the tool module to be tested and the standard input module; establishing the test module; establishing the configuration module and configuring its parameters; and establishing the use case module root directory. If this round of testing requires manual creation of a use case input submodule, manually create a use case input submodule directory and the input file set directories for each use case step within the use case module root directory. Then, manually place the required input files in the folders corresponding to the use case step numbers, ultimately forming the use case input submodule.

[0089] S2. Run the test module and read and store all configuration information of the configuration module by reading the configuration unit.

[0090] S3. Based on the read configuration information, traverse each use case step expected to be executed (use case steps marked as "yes"), test them one by one, and automatically interpret the test results; including the following steps:

[0091] S31. Based on the read configuration information, determine whether to execute the automatic generation of use case step input file set unit to automatically establish the use case step input file set. Specifically, if the automatic generation of use case input submodule is marked as "yes", this unit is executed. Based on the read automatic generation parameters of the current use case step, a use case input submodule directory and a corresponding use case step input file set directory are automatically established (if they already exist, they are not re-established). The required standard input files in the standard input module are then copied to the established use case step input file set directory and overwritten to form the use case step input file set. Otherwise, the above operation is not performed, and the use case step input file set of the use case input submodule manually established in step S1 is used.

[0092] S32. Based on the use case step input file set, the tool module to be tested is called by the execution unit to execute the current use case step and output the execution result; wherein, the current use case step input file set is used as input, the tool module to be tested is run to generate the execution result, a use case output submodule directory is established (if it already exists, it will not be established again) and the corresponding use case step output file set directory under it, and the execution result is placed in the use case step output file set directory; wherein, the execution result includes generated data and log information, and the log information includes a success log and an error log.

[0093] S33. Based on the read configuration information, the judgment unit verifies the execution result and assigns a value to the test success / failure flag according to the test results. The judgment unit compares the result judgment parameters and the corresponding use case step output file set through the assertion method. The verification of the execution result is log information verification or generated data verification. If the execution result is a successful end of the run that meets expectations, then it is determined whether the output success log meets the expected success log in the configuration information. If the execution result is an error exit that meets expectations, then it is determined whether the error statement in the error log of the output execution result meets the expected error log in the configuration information. If the execution result is generated data that meets expectations, then it is determined whether the generated data meets the expected result value in the configuration module. Finally, if the comparison results are completely consistent, the test success / failure flag is assigned a value of "Pass", otherwise it is set to "Fail".

[0094] S4. The reporting unit collects statistics on the test results of step S3 and outputs a test report. The test report may include the total number of test cases, the number of executed test cases, the test case execution rate, the test case pass rate, the number of passed and failed test cases, etc., which can help testers decide whether to conduct regression testing;

[0095] S5. When regression testing is required, a one-click regression test can be performed through the regression test unit; whether to perform regression testing is determined by the tester based on the test report output by the reporting unit. The regression test can choose to use the use case input submodule of the previous round of testing for one-click regression testing, or can modify the automatic generation of the use case input submodule identifier and automatic generation parameters of the configuration module to perform a one-click regression test on the use case input submodule that is automatically rebuilt.

[0096] Below, a specific embodiment is provided to further describe the present invention:

[0097] Step 1: Build an automated testing framework based on the completed test case documentation:

[0098] As attached Figure 2 As shown, the offline tool folder "OfflineTool", the standard input folder "StandardInput", and the automated test framework batch script file Test.bat are placed in the test framework root directory folder; a use case folder "TestCase" and a configuration file "Configuration" are created in the test framework root directory folder. This embodiment does not require manual creation of a use case input submodule.

[0099] Prepare the test framework configuration file: Create an Excel file "testConfig.xlsx". This configuration table contains two sheets. The first sheet is the "BasicInfo" page, which contains the basic information required for this test, including the offline tool folder path, the use case folder path, the standard input folder path, and the option to automatically generate the use case input submodule. Among them, the option to automatically generate the use case input submodule is manually determined by whether the use case input submodule needs to be automatically generated in this round of testing. If so, set this option to "Yes", otherwise set it to "No".

[0100] The second sheet is the "TestCaseInfo" page, which mainly contains the following configuration information:

[0101] Column A: Use case-step number, for example: 0001-step1.

[0102] Column B: Execution ID, which can be used to filter the use case steps that need to be tested in this round.

[0103] (CE is automatically generated parameters)

[0104] Column C: The name of the standard input file that needs to be modified in the use case step. This test framework currently supports file modification in ini, xml, par, and md5 formats.

[0105] Column D: Fields to be modified, write down the position and name of the fields that need to be modified in the standard input file.

[0106] Column E: The target value to which the fields in column D are expected to be modified.

[0107] (Columns FJ are result interpretation parameters, and column FH is the expected data value)

[0108] Column F: The name of the output file corresponding to the execution result.

[0109] Column G: The name of the field in the output file where the execution result is reflected.

[0110] Column H: The expected value of the field.

[0111] Column I: Expected log information. If there are error or success logs in the execution results, the expected log information needs to be entered here in advance.

[0112] Column J: Test success or failure indicator.

[0113] Step 2: Read configuration information: Run the automated test framework batch script file "Test.bat" to read and store all configuration information in the configuration table.

[0114] Step 3: Traverse each use case step marked as "yes" and run the test:

[0115] The automated testing framework batch script file "Test.bat" traverses the use case-step number in column A of the configuration information, generates a use case input submodule folder "Input" under the use case folder "TestCase" (it is created only if it does not exist), and generates a use case step input file set folder named after the current use case-step number under it, and copies the required standard files under the standard input folder "StandardInput" to the use case step input file set folder and modifies them according to the requirements of column CE of the configuration table; currently, this test framework supports 4 types of file rewriting: ini, xml, par and md5, and the md5 value in the md5 file is updated synchronously after rewriting.

[0116] Call the trackside safety platform offline tool batch script file "run.bat" in the offline tool folder "OfflineTool" to execute the current use case step to generate the execution result. Create a use case output submodule folder "Output" in the use case folder "TestCase" (if it already exists, it will not be created again). Create a use case step output file set folder named after the current use case step number under it. Place the generated data files and log files in the use case step output file set folder. The success log file is named "log.log" and the error log file is named "error.log".

[0117] After the current use case step completes, the execution result is automatically checked to see if it meets the expected result in configuration table FH. If the execution result is that the generated data meets the expectations, the field data in the data file corresponding to column FG is determined to meet the expected value in column H. If the expected result is a successful run to completion, the successful end identification statement in "log.log" is checked to see if it meets the expected success log information in column I. If the expected result is an error exit, the error log in "error.log" is checked to see if it meets the expected error log information in column I. Finally, if it meets the expectations, column J is set to "Pass", otherwise it is set to "Fail".

[0118] Step 4. Output the test report: After the run is completed, a new sheet page "TestReport" is created in the configuration table, which will display key information such as the total number of use cases, the number of use cases executed this time, the use case execution rate, the test case pass rate, the details of the passed and failed use case numbers, etc.

[0119] Step 5. Regression test: If the regression test can directly use the use case input submodule of the previous round, or the use case input submodule does not need to be automatically modified and generated, you can ensure that there is a set of input files for each use case step that conforms to the use case steps in the "TestCase" folder. By setting the option of automatically generating use case input submodule on the "BasicInfo" page to "No", and setting the "Execute" in column B of the corresponding use case step on the "TestCaseInfo" page to "Yes", you can then start the automated testing framework batch script file "Test.bat" to run in batches to perform regression testing.

[0120] In summary, the present invention provides an automated testing framework and method, which can rewrite each use case step of the applicable test framework based on the provided standard input data and then perform automated testing in sequence; for complex use case steps in which it is inconvenient to automatically rewrite the input data, the corresponding input data can also be manually prepared and then the automated testing can be performed uniformly; after completing the specified test steps, the pass status of each use case step can be displayed by comparing the expected results with the actual results; a test report can be generated, which includes key information such as the number of test cases and the test pass rate; for cases where the test fails, the test framework can be used for one-click regression later, which provides great convenience for multiple rounds of testing.

[0121] Although the present invention has been described in detail through the above preferred embodiments, it should be understood that the above description is not intended to limit the present invention. After reading the above description, various modifications and substitutions of the present invention will become apparent to those skilled in the art. Therefore, the scope of protection of the present invention should be defined by the appended claims.

Claims

1. An automated testing framework for performing software testing on a trackside safety platform offline tool based on pre-written test case documents, wherein the test case documents include multiple use cases, and the software test is composed of tests of each use case; characterized in that: The automated testing framework includes: The tool module to be tested is the offline tool of the trackside safety platform; The standard input module is a collection of all standard input files that can be used for the successful operation of the tool module under test; The use case module includes a use case input submodule and a use case output submodule; wherein the use case input submodule is a collection of input files required to execute each use case, which is manually created or automatically generated based on the standard input module; the use case output submodule is a collection of output files generated after executing each use case; Configuration module, used to set and save parameter configuration of the automated testing framework; A test module, configured to perform automated testing on the tool module under test based on the use case input submodule and the configuration module of the use case module, and generate a use case output submodule; wherein the test module can automatically generate the use case input submodule based on the configuration module and the standard input module; The use case input submodule includes a plurality of use case step input file sets, each of which is a collection of one or more use case step input files required to execute the use case step; The test module includes a unit for automatically generating a use case step input file set, which is capable of automatically generating a use case step input file set based on a configuration module and a standard input module, and each of the use case step input file sets constitutes a use case input submodule; the configuration module includes a basic information configuration unit and a use case configuration unit; wherein the basic information configuration unit includes an automatic generation of a use case input submodule identifier for indicating whether to automatically generate the use case input submodule; The use case configuration unit includes a plurality of use case step configuration subunits for configuring relevant parameters for executing each use case step, each of the use case step configuration subunits corresponding to each use case step; each of the use case step configuration subunits includes automatic generation parameters for automatically generating a use case step input file set; The automatic generation of use case step input file set unit automatically generates the use case step input file set based on the automatic generation of use case input submodule identifier and automatic generation parameters.

2. The automated testing framework according to claim 1, wherein: Each use case step input file of the use case step input file set is an input file that is manually prepared and conforms to the use case step, or is a standard input file that is copied and rewritten from the standard input module.

3. The automated testing framework according to claim 2, wherein: The automatically generated parameters include the copy file name, rewrite field name and rewrite target value, which are respectively used to indicate the name of the standard input file that needs to be copied and rewritten, the name of the field that needs to be rewritten in the standard input file, and the rewrite target value of the field.

4. The automated testing framework according to claim 3, wherein: Whether the automatic use case generation step input file set unit is executed or not is determined based on the read automatic use case generation input submodule identifier; When the automatic generation of use case step input file set unit is running, it automatically establishes a use case input submodule directory and the corresponding use case step input file set directory under it based on the automatic generation parameters of the current use case step, and then copies the required standard input files in the standard input module to the established use case step input file set directory and rewrites it to form the corresponding use case step input file set.

5. The automated testing framework according to claim 1, wherein: The test module includes an execution unit, which is used to call the tool module to be tested to execute each use case step based on the input file set of each use case step, and output the execution result.

6. The automated testing framework according to claim 5, wherein: The use case output submodule includes a plurality of use case step output file sets, each of which is a collection of one or more use case step output files output by executing the use case step; The output execution result of the execution unit includes establishing a use case output submodule directory and a corresponding use case step output file set directory thereunder, and placing the execution result into the use case step output file set directory.

7. The automated testing framework according to claim 5, wherein: Each of the use case step configuration subunits includes: Use case-step number, used to indicate the use case step corresponding to the configuration subunit of this use case step; Execution flag, used to indicate whether this use case step is executed; The execution unit determines whether to execute the use case step corresponding to the use case step number based on the execution identifier.

8. The automated testing framework according to claim 6, wherein: Each of the use case step configuration subunits includes result interpretation parameters, wherein the result interpretation parameters include an expected result, which is an expected execution result after executing the use case step manually set and used as a basis for interpreting the execution result; The test module includes a judgment unit for judging the expected result of the parameter based on the result judgment, comparing it with the execution result in the corresponding output file set of the use case step, and automatically checking whether the execution result is correct.

9. The automated testing framework according to claim 8, wherein: The judgment unit compares the execution result and the result judgment parameter through an assertion method.

10. The automated testing framework according to claim 8, wherein: The execution result of the execution unit includes log information, or generates data and log information; The result interpretation parameters include expected data values and expected log information; wherein, if the execution result is that the generated data meets expectations, the expected data value is set, which is the expected generated data value of each result field in the output file of each use case step after executing the use case step; if the execution result is that the running ends successfully or the error exit meets expectations, the expected log information is set, which is the log expected to be generated after executing the use case step; The inspection of the execution result by the judgment unit is a log information inspection or a generated data inspection, wherein the log information inspection is performed when the expected result of the use case step is a successful completion of the operation or an error exit, and the log information in the execution result is compared with the expected log information in the result judgment parameter; the generated data inspection is performed when the expected result of the use case step is that the generated data meets expectations, and the generated data in the execution result is compared with the expected data value in the result judgment parameter.

11. The automated testing framework according to claim 8, wherein: The result interpretation parameters include a test success / failure flag, which is used to indicate whether the test of this use case step has passed. The interpretation unit automatically fills in the flag according to the interpretation execution result.

12. The automated testing framework according to claim 10, wherein: The expected data value is an exact value, a range, an enumeration or a regular expression.

13. The automated testing framework according to claim 8, wherein: The test module includes a reporting unit for collecting test statistics and outputting a test report after completing the current round of testing.

14. The automated testing framework according to claim 13, wherein: The test report includes part or all of the details of the total number of use cases, the number of executed use cases, the use case execution rate, the test case pass rate, the passed use case number and the failed use case number.

15. The automated testing framework according to claim 13, wherein: The test module includes a regression test unit for performing one-click regression testing.

16. The automated testing framework according to claim 15, wherein: The regression test selects to use the use case input submodule of the previous round of testing for one-click regression testing, or to perform one-click regression testing of the automatically reconstructed use case input submodule by modifying the automatically generated use case input submodule identifier and automatically generated parameters of the configuration module.

17. The automated testing framework according to claim 1, wherein: The basic information configuration unit also includes module path information, which is used to set the root path information of each module.

18. The automated testing framework according to claim 1, wherein: The test module further includes a configuration reading unit, which is used to read and store configuration information in the configuration module and provide it to other units of the test module for use.

19. An automated testing method, characterized in that: The automated testing framework according to any one of claims 1 to 18 is implemented, comprising the steps of: S1. Build an automated testing framework that includes at least a tool module to be tested, a standard input module, a configuration module, and a test module; S2. Run the test module and read and store all configuration information of the configuration module by reading the configuration unit; S3. Based on the read configuration information, traverse each expected use case step and test it one by one, and automatically interpret the test results.

20. The automated testing method according to claim 19, wherein: Step S1 includes: Import the tool module to be tested and the standard input module; establish a test module; establish a configuration module and configure its parameters; establish the root directory of the use case module.

21. The automated testing method according to claim 20, wherein: If this round of testing requires manual creation of a use case input submodule, step S1 further includes: Manually create a use case input submodule directory and each use case step input file set directory under the use case module root directory, and then manually create each use case step input file and place it in each corresponding use case step input file set directory to form a use case step input file set. Each use case step input file set constitutes a use case input submodule.

22. The automated testing method according to claim 19, wherein: Step S3 includes: S31. Based on the read configuration information, determine whether to execute the automatic generation of the use case step input file set unit to automatically establish the use case step input file set; S32. Based on the input file set of the use case step, the execution unit calls the tool module to be tested to execute the current use case step and outputs the execution result; S33. Based on the read configuration information, the unit test execution result is judged, and a test success / failure flag is assigned according to the test result.

23. The automated testing method according to claim 19, wherein: Step S3 is followed by step S4: a reporting unit collects statistics on the test results of step S3 and outputs a test report.

24. The automated testing method according to claim 23, wherein: Step S4 is followed by step S5: when regression testing is required, a one-click regression test is performed by the regression testing unit.

Images