A Method for Stable Generation of Module-Level Simulink Test Cases

Abstract

The present invention discloses a method for stably generating module-level Simulink test cases, including: constructing a software module to be tested; sorting all the smallest unit standard modules in the official module library of the software to be tested; generating all modules from the official module library of the software to be tested; running the test case to be tested, and deleting the module link group that causes compilation failure if the compilation fails; recording the information of the module link group in the test case to be tested that runs successfully into the analysis database of the software module to be tested; repeating the above operations until all the smallest unit standard modules in the official module library of the software to be tested are traversed; in the stage of randomly generating test cases, randomly generating an initial module that can generate signals from the official module library of the software to be tested, inputting the information of the module into the analysis database of the software module to be tested to query its link situation, running the generated test case, saving it if the running is successful, discarding it if the compilation fails, and counting the generation rate of the test case to be tested.

Description

Technical Field

[0001] The present invention relates to the field of software testing, and in particular to a method for stably generating module-level Simulink test cases. Background Art

[0002] As a commercial CPS development toolchain, Simulink has been widely used to design, simulate, and generate embedded code for CPS models in many safety-critical applications. In the Simulink development toolchain, users design Simulink models as block diagrams. It analyzes, compiles, and executes CPS models before transferring them to hardware to simulate the behavior of CPS. When the simulation meets the user's expectations, Simulink automatically generates embedded code and deploys it to the target application. The above development process allows users to quickly build CPS prototypes and deploy applications without writing code. And the Simulink compiler, as the most important part of the Simulink development toolchain, will have unthinkable consequences if errors occur. To reduce the probability of errors in the Simulink compiler, current tests for the Simulink compiler are gradually emerging.

[0003] Currently, in the existing Simulink development toolchain, the method for generating Simulink test case models is represented by SLforge, which is based on semi-formal Simulink model specifications to guide random model generation and machine learning-based random generation. Its disadvantage is that a complete generation specification needs to be provided to improve the success rate of model generation. At the same time, because these methods refer to existing real models, the generated styles mainly depend on the existing models, and the generated styles of the models have limitations. Summary of the Invention

[0004] According to the problems existing in the prior art, the present invention discloses a method for stably generating module-level Simulink test cases, which specifically includes the following steps:

[0005] Construct a software module under test, and analyze the database for storing the linking situation of each minimum unit standard module in the software under test;

[0006] Sort all the minimum unit standard modules in the official module library of the software under test, and extract non-repeated module A from them, and copy and generate multiple module A so that the number of module A is equal to the number of all minimum unit standard modules in the software under test;

[0007] Generate all modules from the official module library of the software under test, and link each module to the module A in the test case correspondingly, so that a non-repeated module is chained after each module A to form a module link group;

[0008] Run the test case under test. If the compilation fails, delete the module link group that causes the compilation failure.

[0009] Record the information of the module link group in the test case under test that runs successfully into the analysis database of the software module under test.

[0010] Repeat the above operations until all the minimum unit standard modules in the official module library of the software under test are traversed.

[0011] In the stage of randomly generating test cases, randomly generate an initial module that can generate signals from the official module library of the software under test, input the information of this module into the analysis database of the software module under test to query its link situation, and randomly select linkable modules from them for generation, and so on to generate test cases under test.

[0012] Run the generated test cases. If the run is successful, save them; if the compilation fails, discard them, and count the generation rate of the test cases under test.

[0013] The link situation of each minimum unit standard module in the analysis database of the software module under test is stored in the form of a database table, and the main information recorded includes the name information of the source module and the name information of the destination module.

[0014] The minimum unit standard module is a module that cannot be further split and is given in the official module library. The initial module is a signal generation module, which is some modules that can generate signals in the software under test.

[0015] The link method between module A and other modules is to place module A on the side closer to the signal input, and link the input port of other modules to the output port of module A through a signal line.

[0016] The source module is the module on the signal inflow side or closer to the signal inflow side during linking, and the destination module is the module on the information outflow side or closer to the signal outflow side.

[0017] Due to the adoption of the above technical solution, a method for stably generating Simulink test cases at the module level provided by the present invention performs a large number of combined compilations for each module in the early stage, screens out successfully compilable module combinations and records them, improving the deficiency that the Simulink of the software under test lacks effective language specifications and thus cannot stably guide the generation of models. This method guides the generation of test cases for the software under test in the subsequent stage, greatly improving the overall compilation success rate after the test cases are generated, which is of great significance for the generation of the software model under test, and also provides a large number of effective and available test case sets for the subsequent work of testing the software under test. Description of the Drawings

[0018] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the accompanying drawings required for the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only some embodiments recorded in the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0019] Figure 1 It is the flowchart of the method of the present invention in the present invention;

[0020] Figure 2 It is the schematic diagram of module batch generation and linking in the present invention;

[0021] Figure 3 It is the schematic diagram of model generation in the present invention. Detailed implementation manners

[0022] To make the technical solutions and advantages of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention:

[0023] As Figure 1 shown is the flowchart of a method for stably generating module-level Simulink test cases. The specific solution is as follows:

[0024] In step S101: Construct an analysis database for the software module under test;

[0025] The construction of the analysis database for the software module under test can be achieved by using, but not limited to, Mysql. After building an empty database, create a database table. There are three fields in the database table, namely the primary key, the information of the departure module, and the information of the arrival module. Taking Simulink as an example, the generation of modules can use a dedicated official API. Usually, there are multiple ways. For example, if the name of the module to be generated is known, then for the information of the departure module and the arrival module in the software under test, they can be the specific names of the modules. The definitions of the departure module and the arrival module are that the departure module usually obtains signals faster than the arrival module, and the signals flow into the input port of the arrival module through the output port of the departure module. The departure module and the arrival module are connected by one or more signal lines.

[0026] In subsequent steps, if it is necessary to add the analysis data of the software module under test to the analysis database for the software module under test, it needs to comply with the above specifications, that is, collect the information of the departure module and the arrival module. The analysis database for the software module under test will not allow the addition of duplicate information to ensure that each module can be generated with equal probability in subsequent steps.

[0027] In step S102: Module batch generation and link verification;

[0028] Specifically, in this step, all the smallest unit standard modules in the official module library of the software under test are sorted, and non-repetitive modules are extracted from them. Multiple copies of each module are generated so that the number of modules is equal to the number of all the smallest unit standard modules in the software under test. Then, all the modules are generated from the official module library of the software under test, and each module is linked to the corresponding module in the test case. After that, a non-repetitive module is linked to each module to form a module link group. As Figure 2 shown, module A 21 is a basic module randomly selected from the official module library of the software under test by this method. This module is not composed of other basic modules and performs specific tasks in the model. By copying the information of this module multiple times, multiple identical modules such as module A 23 and module A 25 are generated. The number of modules is equal to the total number of modules in the official module library of the software under test, which is used for one-to-one matching in the subsequent link verification phase. Then, all the modules are generated from the official module library of the software under test. It should be noted that all the modules are generated only once. The generated modules are connected to module A. This method gives an example in the figure. Module B 22, module C 24, and module D 26 are a microcosm of all the modules, and they represent the modules that perform various different functions in the software under test (it is also possible to generate module A, which is allowed). These subsequently generated modules are linked to the batch-generated module A one by one. Each group contains only two modules, namely module A and other modules. It should be emphasized that some modules have no input ports and cannot support linking with module A. This method will automatically delete these modules and label them as signal-emitting modules. For modules without output ports, this method calls them signal-terminating modules. When the generated module A meets the requirements of the signal-terminating module in the subsequent process, we will skip it and not discuss it.

[0029] In step S103: Collect the analysis data of the modules of the software under test;

[0030] After completing the above steps, run the test case. At this time, if the operation is normal, the existing link information in the test case is recorded in the module analysis database of the software under test. If the operation fails and error information is generated, the module link group that causes the error information is deleted until the operation is normal and no error information is generated. The concepts of normal operation and error information will be standardized and described in step S105. The information that needs to be recorded in the module analysis database of the software under test has also been described in step S101, so no further elaboration will be made here.

[0031] Steps S102 and S103 need to be performed on all the modules in the official module library of the software under test once and recorded;

[0032] In step S104: Random generation of the model;

[0033] Currently, the analyzed database of the software module under test should have recorded the module link information of all modules in the official module library of the software under test, and after verification through operation, it can be confirmed that the links between modules meet the requirements of the software under test, which theoretically provides a feasible and effective specification for the accuracy of random model generation. In the random model generation stage, first randomly find a signal-emitting module in the official module library of the software under test (specifically referring to a module that can independently generate other signals such as linear and non-linear signals. This module has the characteristic that it has no input ports and is the direct or indirect source of signals from other modules), and input the information of this signal-emitting module into the starting module field in the analyzed database table of the software module under test to search for the content in the arriving module field. We perform random sampling on the randomly found module fields according to the maximum number of output ports supported by the current signal-emitting module. The number of newly generated modules should be less than or equal to the maximum number of output ports of the signal-emitting module. In addition, the random sampling method is the same as the random sampling mentioned in other methods, without special operations. As Figure 3 shown, module A31 is the signal-emitting module. According to the maximum number of its output ports, we randomly generated other modules such as module B 32, module C 33, and module D 34, and linked them to module A31 through direct links. After that, we also input the information of modules B 32, C 33, D 34, etc. into the starting module field in the analyzed database of the software module under test to search, generate according to the above method, and link the generated modules after modules B 32, C 33, D 34, and so on in a cycle, presenting a tree-like generation method. Until the generated quantity reaches the required number in the test cases of this method (this number is usually set by the engineer himself).

[0034] In step S105: Model operation and recording;

[0035] Specifically, run the randomly generated test case model in various simulation modes of the software under test (especially Simulink). Collect and record information such as the name and running time of the models that can run normally without generating error messages. Finally, calculate the random generation probability of the test case model. The calculation method is the number of test case models that can run normally divided by the total number of randomly generated test case models. It should be emphasized that the error messages cover the prompt messages popped up when the software under test crashes due to the test cases, the compilation messages popped up when the generated test case model does not meet the specifications of the software under test and causes compilation failure, and also cover various situations where the test case runs out of time and cannot terminate, and the software under test is forced to close. The determination of normal operation in this method is that it can run normally and end within a given time. The specific given time is usually 10s.

[0036] As mentioned above, it is only the preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention, according to the technical solution and inventive concept of the present invention, making equivalent substitutions or changes should be covered within the protection scope of the present invention.

Claims

1. A method for stably generating module-level Simulink test cases, characterized in that Including: Construct the software module under test, and analyze the database for storing the linking status of each minimum unit standard module in the software under test; Sort all the minimum unit standard modules in the official module library of the software under test, extract the non-repeated module A from them, and copy and generate multiple module As so that the number of module As is equal to the number of all minimum unit standard modules in the software under test; Generate all modules from the official module library of the software under test, and link each module to the module A in the test case so that a non-repeated module is linked after each module A to form a module link group; Run the test case. If the compilation fails, delete the module link group that causes the compilation failure; Record the information of the module link group in the test case that runs successfully into the software module analysis database of the software under test; Repeat the above operations until all the minimum unit standard modules in the official module library of the software under test are traversed; In the stage of randomly generating test cases, randomly generate an initial module that can generate signals from the official module library of the software under test, input the information of this module into the software module analysis database of the software under test to query its linking status, and randomly select a linkable module from it for generation, and so on to generate test cases; Run the generated test cases. If the run is successful, save them. If the compilation fails, discard them, and count the generation rate of the test cases; The minimum unit standard module is a module that cannot be further split and is given in the official module library. The initial module is a signal generation module, which is some modules that can generate signals in the software under test.

2. A method for stably generating module-level Simulink test cases according to claim 1, characterized in that: The linking status of each minimum unit standard module in the software module analysis database of the software under test is stored in the form of a database table, and the information recorded includes the name information of the starting module and the name information of the arriving module.

3. A method for stably generating module-level Simulink test cases according to claim 1, characterized in that: The linking method between module A and other modules is to place module A on the side closer to the signal input, and link the input port of other modules to the output port of module A through a signal line.

4. A method for stably generating module-level Simulink test cases according to claim 1, characterized in that: The starting module is the module on the signal inflow side or closer to the signal inflow side during linking, and the arriving module is the module on the information outflow side or closer to the signal outflow side.

Images