An automated simulation testing method through test requirement modeling

Abstract

An automated simulation test method for test requirement modeling, comprising: modeling modules of a system under test using a module definition graph, and defining parameters, behaviors of the modules, and whether to use existing model files. A parameter graph is automatically generated from the module definition graph, the parameter graph containing models and ports corresponding to the modules participating in simulation. Input ports and output ports between the models in the parameter graph are connected according to variable transmission relationships of the modules. A state machine graph is modeled according to test requirements of the system. Test cases are automatically generated according to parameter definitions of the module definition graph and states and transition conditions in the state machine graph. Simulation engineering parameters are defined, model files are generated or existing model files are imported according to the module definition graph, and a model connection graph is generated according to the parameter graph. The engineering files are imported into simulation tool software to build a simulation engineering. The test cases are executed, and simulation results are collected. The present application improves the accuracy and efficiency of the test.

Description

Technical Field

[0001] This invention relates to an automated simulation testing method based on test requirements modeling, belonging to the field of computer technology. Background Technology

[0002] Currently, significant progress has been made in the research of automated testing in the testing field, improving testing efficiency and coverage, but some shortcomings still exist:

[0003] 1) Disconnect between automated testing and system design: Traditional automated testing rarely integrates directly with the system design phase. This means that testers can only begin writing test cases and executing tests after the system design is complete, thus extending the development cycle. Furthermore, it fails to guarantee consistency between the testing process and the system design.

[0004] 2) Expression and understanding of test requirements: Existing methods often lack an effective way to express complex test requirements, especially system-level test requirements. This leads to incomplete testing and inaccurate test cases. Test requirement modeling often remains at the level of providing information for testers or automatically generating test cases, making it difficult to directly participate in the automated simulation testing process.

[0005] 3) Insufficient adaptability and flexibility: As software systems evolve, test cases also need to be constantly updated to match new requirements. Existing automated testing methods often struggle to adapt quickly to these changes, leading to decreased test coverage.

[0006] The above shortcomings prevent automated simulation testing from being achieved simply by modeling test requirements, making it difficult to further improve the accuracy and efficiency of testing. Summary of the Invention

[0007] The technical problem solved by this invention is to overcome the shortcomings of the prior art and provide an automated simulation testing method that models test requirements. By using SysML to fully model test requirements, test cases and project files (including model files, connection relationships, and simulation parameter information) required for automated simulation testing can be automatically generated, and simulation tools can be called to realize automated simulation testing.

[0008] The technical solution of this invention is: an automated simulation testing method based on test requirement modeling, comprising:

[0009] S-1 models the system under test to obtain a module definition diagram of the system under test. The module definition diagram describes the relationship between the modules in the system, and describes whether the module participates in the test simulation, the parameters of the module, and the behavior of the module. At the same time, the module definition diagram also contains simulation parameter information.

[0010] S-2 automatically generates parameter diagrams through module definition diagrams. The parameter diagrams describe the variable transfer relationships of each module participating in the simulation test in the system under test. The parameter diagrams include the models, input ports, and output ports corresponding to the modules participating in the simulation.

[0011] In the parametric diagram of S-3, the input and output ports between models are manually connected according to the variable transfer relationship between modules;

[0012] S-4 models the test requirements of the system under test and obtains a state machine diagram based on the test requirements, including states and state transition conditions.

[0013] S-5 automatically generates test cases based on the parameter definitions of modules in the module definition diagram and the states and transition conditions in the state machine diagram.

[0014] S-6 defines simulation engineering parameters;

[0015] S-7 uses the module definition diagram to call the modeling tool to generate model files or import existing model files, and generates model connection diagrams based on parameter diagrams.

[0016] S-8 uses model files, model connection diagrams, and simulation parameters to form project files, which are then imported into simulation tools to build simulation projects.

[0017] S-9 executes test cases;

[0018] The S-10 collects, records, analyzes, and displays simulation results.

[0019] Preferably, in S-1, the parameters of the module include: input parameters, output parameters, and internal parameters;

[0020] The module's behavior includes: initialization behavior, iteration behavior, and whether to use an existing model file;

[0021] The simulation parameter information includes the range of each simulation parameter.

[0022] Preferably, the simulation engineering parameters in S-6 include the simulation step size.

[0023] Preferably, the test cases to be executed include:

[0024] Assign values ​​to the simulation parameters;

[0025] Set the test inputs, including initial parameters and parameter modifications during the simulation process;

[0026] Simulation control, including starting and stopping the simulation;

[0027] Judging from simulation results.

[0028] Secondly, an automated simulation testing system based on test requirement modeling is provided, characterized by comprising: a test requirement modeling unit, a test case generation unit, a project file generation unit, and a test case execution analysis unit; wherein:

[0029] The test requirement modeling unit is used to model the system under test (SUT), including a module definition subunit, a parameter definition subunit, and a state description subunit. The module definition subunit models the SUT, obtains a module definition diagram, outputs it to the parameter definition subunit, outputs the simulation parameters in the module definition diagram to the project file generation unit, and outputs the parameter definitions of the modules in the module definition diagram to the test case generation unit. The parameter definition subunit automatically generates a parameter diagram based on the module definition diagram and outputs it to the project file generation unit. The state description subunit models the test requirements of the SUT, generates a state machine diagram based on the test requirements, and outputs it to the test case generation unit.

[0030] The test case generation unit automatically generates test cases based on the parameter definitions of the modules in the module definition diagram and the states and transition conditions in the state machine diagram, and outputs them to the test case execution analysis unit.

[0031] The engineering file generation unit defines simulation engineering parameters, calls modeling tools to generate model files based on the module definition diagram or imports existing model files, generates model connection diagrams based on parameter diagrams, uses model files, model connection diagrams and simulation parameters to form engineering files, and outputs them to the test case execution and analysis unit.

[0032] The test case execution and analysis unit executes test cases, collects simulation results, and records, analyzes, and displays them.

[0033] Preferably, after the parameter definition subunit automatically generates the parameter diagram based on the module definition diagram, the input and output ports between the models are manually connected according to the variable transfer relationship between the modules.

[0034] Preferably, the module definition diagram generated by the test requirement modeling unit is used to describe the relationships between various modules in the system under test, and to describe whether a module participates in the test simulation, the module's parameters, and the module's behavior; the module definition diagram also includes simulation parameter information; wherein:

[0035] The parameters of a module include: input parameters, output parameters, and internal parameters;

[0036] The module's behavior includes: initialization behavior, iteration behavior, and whether to use an existing model file;

[0037] The simulation parameter information includes the range of each simulation parameter.

[0038] Preferably, when the test case execution analysis unit executes test cases:

[0039] Assign values ​​to the simulation parameters;

[0040] Set the test inputs, including initial parameters and parameter modifications during the simulation process;

[0041] Simulation control, including starting and stopping the simulation;

[0042] Judging from simulation results.

[0043] Compared with the prior art, the present invention has the following advantages:

[0044] (1) This invention only requires modeling the test requirements to achieve automated testing of the system under test without additional manual intervention. Since SysML is frequently used in system design, test requirements modeling can often be performed in the design phase, thereby ensuring the consistency between testing and design and improving the accuracy of testing.

[0045] (2) Since the present invention requires less human intervention, there is no need to redesign the test and update the test cases in the regression test. Only the test requirement model needs to be updated to automatically complete the regression test, which greatly improves the efficiency of the test. Attached Figure Description

[0046] Figure 1 A schematic diagram illustrating the test requirement modeling and automated test simulation method of the present invention;

[0047] Figure 2 This is a flowchart illustrating the application of the automated testing simulation method of the present invention.

[0048] Figure 3 This is a schematic diagram illustrating the module definition in an embodiment of the present invention;

[0049] Figure 4 This is a parameter diagram of an embodiment of the present invention;

[0050] Figure 5 This is a schematic diagram of the state machine according to an embodiment of the present invention. Detailed Implementation

[0051] This invention provides the following technical solution: an automated simulation testing method based on test requirement modeling. This method uses SysML (System Modeling Language) to fully model test requirements, automatically generating test cases and project files (including model files, connection relationships, and simulation parameter information) required for automated simulation testing, and then calling simulation tools to achieve automated simulation testing. Figure 1 As shown, the test requirement model includes the following parts:

[0052] 1) The module definition diagram describes the relationship between the modules in the system, and describes whether the module participates in the test simulation, the parameters of the module and the behavior of the module. The module definition diagram also includes simulation parameter information, and the simulation parameter information in the initial state gives the range of each simulation parameter.

[0053] 2) The parameter diagram describes the variable transfer relationships among the modules involved in the simulation test within the system; that is, it represents the connection relationship diagram of the model in the simulation test. The models and ports in the parameter diagram are automatically generated from the module definition diagram.

[0054] 3) Model the system's test requirements using state machine diagrams, and describe the test judgment criteria in the test using transition conditions and states.

[0055] After completing the test requirements modeling, test project files and test cases are generated automatically, and the simulation project is called to achieve automated simulation testing. The project files include model files, connection relationships, and simulation parameter information.

[0056] 1) Model files are automatically generated through module definition diagrams. Modules in the module definition diagram contain the model's parameters and behavioral descriptions, which can be translated into a modeling language and used with modeling tools to generate model files. Alternatively, existing model files corresponding to modules can be directly specified in the module definition diagram for use during testing and simulation.

[0057] 2) The connection relationships are generated through the parametric diagram.

[0058] 3) Test cases are generated using state machine diagrams.

[0059] like Figure 2 As shown, the process of this automated testing method includes:

[0060] (1-1) Use a module definition diagram to model the modules of the system under test, and define whether the module's parameters (input parameters, output parameters, internal parameters), behaviors (initialization behaviors and iterative behaviors) participate in the simulation, and whether to use an existing model file.

[0061] (1-2) The parameter diagram is automatically generated from the module definition diagram, which already contains the model and port corresponding to the module participating in the simulation.

[0062] (1-3) In the parametric diagram, the input and output ports between models are manually connected according to the variable transfer relationship between modules.

[0063] (1-4) Use state machine diagrams to model the test requirements of the system under test.

[0064] (1-5) Test cases are automatically generated based on the parameter definitions of the modules in the module definition diagram and the states and transition conditions in the state machine diagram.

[0065] (1-6) Define the simulation engineering parameters (such as simulation step size).

[0066] (1-7) Generate model files by calling the modeling tool based on the module definition diagram or import existing model files, and generate model connection diagrams based on the parameter diagram.

[0067] (1-8) Import the project files (model files, model connection diagrams, simulation parameter information) into the simulation tool software to build the simulation project.

[0068] (1-9) Execute test cases. The specific steps include setting simulation parameters, setting test inputs (including initial parameters and parameter modifications during the simulation process), simulation control (starting simulation and stopping simulation), and judging simulation results.

[0069] (1-10) Collect simulation results, record, analyze and present them.

[0070] Example:

[0071] The invention's concept and technical effects are illustrated by testing the temperature control function of a temperature control system.

[0072] First, the test target temperature control system is modeled using a module definition diagram. Here, a simplified model is used to model the temperature control system as follows: Figure 3 As shown, the Unreal prefix indicates that the module will not be used as a digital model in simulation testing. Figure 3 The intermediate temperature control system is a general concept comprising two modules: a heater and an ambient temperature module. Therefore, neither module needs to participate in the simulation. Both the heater and ambient temperature modules are prefixed with "Real," indicating that they will be used as digital models in the simulation. Modules participating in the simulation will have two parts: values ​​and operations (behaviors). The value part has three types: input ports, output ports, and parameters, corresponding to the parameter types in model-based systems engineering, prefixed with "Input," "Output," and "Parameter" for software identification. The operation part allows defining module functions, which can participate in test case generation and model generation. In the example, this function calculates the heating power output of the heater model and the temperature of the ambient temperature model.

[0073] After modeling the temperature control system using a module definition diagram, the software can read the model and port information to generate a model and port information module definition diagram. Figure 1 The parameter diagram, such as Figure 4 As shown, the tester can then connect the input and output ports of the model in the parametric diagram according to the internal relationships of the test target.

[0074] Once the test requirements are defined, they can be modeled using a state machine diagram. Figure 5 The example demonstrates the testing requirements for the temperature control function of a temperature control system. The parameters used in the modeling are ensured to be consistent with the module definition diagram via software, avoiding errors. The temperature control system in the example has two states: heating and not heating. When the power switch is turned on, the system enters either the heating or not heating state based on a comparison between the target temperature of the heater model and the ambient temperature model. In the heating state, the heater's heating power output is adjusted to 1000; in the not heating state, the heater's heating power output is adjusted to 0. A check is performed every 20 seconds: if the ambient temperature model's temperature is greater than or equal to the target temperature of the heater model, the system enters the not heating state; otherwise, it enters the heating state. An algorithm can parse a state transition path from the start state to the stop state. By assigning initial values ​​to the model's parameters and the transition conditions in the initial path using a specific algorithm, test cases are obtained.

[0075] After completing the above modeling using the test requirements modeling software of this invention, model files can be automatically generated by calling the modeling tool through the module definition diagram, or existing model files corresponding to modules can be directly specified in the module definition diagram. In this example, only two model files will be generated: heater and ambient temperature. Then, the connection relationships in the parameter diagram are extracted, and test cases are generated using a certain algorithm. Finally, the simulation project parameters are defined in the test requirements modeling software. When the user operates the test requirements modeling software to execute the test, the project files (model files, model connection diagrams, simulation parameter information) will be imported into the simulation tool software, the simulation project will be built, and the test cases will be executed. For example, according to... Figure 4 A generated test case will be executed according to the following process:

[0076] 1) Set initial parameters: target temperature 25℃, initial temperature 20℃;

[0077] 2) Start the simulation;

[0078] 3) Set the heater's power switch port to 1;

[0079] 4) Assume the ambient temperature port < the heater's target temperature parameter;

[0080] 4) Assume that the heating power output port of the heater is 1000;

[0081] 5) After 20 seconds, assert that the ambient temperature at the temperature port is greater than or equal to the target temperature parameter of the heater.

[0082] 6) Assert whether the heating power output port of the heater is 0;

[0083] 7) Set the heater's power switch port to 0;

[0084] 8) Simulation ends.

[0085] The contents not described in detail in this specification are existing technologies known to those skilled in the art.

Claims

1. An automated simulation testing method based on test requirement modeling, characterized in that... include: S-1 models the system under test to obtain a module definition diagram of the system under test. The module definition diagram describes the relationship between the modules in the system, and describes whether the module participates in the test simulation, the parameters of the module, and the behavior of the module. At the same time, the module definition diagram also contains simulation parameter information. S-2 automatically generates parameter diagrams through module definition diagrams. The parameter diagrams describe the variable transfer relationships of each module participating in the simulation test in the system under test. The parameter diagrams include the models, input ports, and output ports corresponding to the modules participating in the simulation. In the parametric diagram of S-3, the input and output ports between models are manually connected according to the variable transfer relationship between modules; S-4 models the test requirements of the system under test and obtains a state machine diagram based on the test requirements, including states and state transition conditions. S-5 automatically generates test cases based on the parameter definitions of modules in the module definition diagram and the states and transition conditions in the state machine diagram. S-6 defines simulation engineering parameters; S-7 uses the module definition diagram to call the modeling tool to generate model files or import existing model files, and generates model connection diagrams based on parameter diagrams. S-8 uses model files, model connection diagrams, and simulation parameters to form project files, which are then imported into simulation tools to build simulation projects. S-9 executes test cases; The S-10 collects, records, analyzes, and displays simulation results.

2. The automated simulation testing method based on test requirement modeling according to claim 1, characterized in that: In S-1, the module's parameters include: input parameters, output parameters, and internal parameters; The module's behavior includes: initialization behavior, iteration behavior, and whether to use an existing model file; The simulation parameter information includes the range of each simulation parameter.

3. The automated simulation testing method based on test requirement modeling according to claim 1, characterized in that: The simulation engineering parameters in S-6 include the simulation step size.

4. The automated simulation testing method based on test requirement modeling according to claim 1, characterized in that: Executing test cases includes: Assign values ​​to the simulation parameters; Set the test inputs, including initial parameters and parameter modifications during the simulation process; Simulation control, including starting and stopping the simulation; Judging from simulation results.

5. An automated simulation testing system based on test requirements modeling, characterized in that... include: The test requirements modeling unit, test case generation unit, project file generation unit, and test case execution analysis unit are included; among them: The test requirement modeling unit is used to model the system under test (SUT), including a module definition subunit, a parameter definition subunit, and a state description subunit. The module definition subunit models the SUT, obtains a module definition diagram, outputs it to the parameter definition subunit, outputs the simulation parameters in the module definition diagram to the project file generation unit, and outputs the parameter definitions of the modules in the module definition diagram to the test case generation unit. The parameter definition subunit automatically generates a parameter diagram based on the module definition diagram and outputs it to the project file generation unit. The state description subunit models the test requirements of the SUT, generates a state machine diagram based on the test requirements, and outputs it to the test case generation unit. The test case generation unit automatically generates test cases based on the parameter definitions of the modules in the module definition diagram and the states and transition conditions in the state machine diagram, and outputs them to the test case execution analysis unit. The engineering file generation unit defines simulation engineering parameters, calls modeling tools to generate model files based on the module definition diagram or imports existing model files, generates model connection diagrams based on parameter diagrams, uses model files, model connection diagrams and simulation parameters to form engineering files, and outputs them to the test case execution and analysis unit. The test case execution and analysis unit executes test cases, collects simulation results, and records, analyzes, and displays them.

6. The automated simulation testing system based on test requirement modeling according to claim 5, characterized in that: After the parameter definition subunit automatically generates the parameter diagram based on the module definition diagram, the input and output ports between the models are manually connected according to the variable transfer relationship between the modules.

7. An automated simulation testing system based on test requirement modeling as described in claim 5, characterized in that: The module definition diagram generated by the test requirements modeling unit is used to describe the relationships between various modules in the system under test, and to describe whether a module participates in the test simulation, the module's parameters, and the module's behavior. The module definition diagram also includes simulation parameter information; among which: The parameters of a module include: input parameters, output parameters, and internal parameters; The module's behavior includes: initialization behavior, iteration behavior, and whether to use an existing model file; The simulation parameter information includes the range of each simulation parameter.

8. An automated simulation testing system based on test requirement modeling according to claim 5, characterized in that: When the test case execution analysis unit executes test cases: Assign values ​​to the simulation parameters; Set the test inputs, including initial parameters and parameter modifications during the simulation process; Simulation control, including starting and stopping the simulation; Judging from simulation results.

Images