Method, apparatus and electronic device for generating test cases

Abstract

The application relates to a method, device and electronic equipment for generating test cases. The method for generating test cases comprises the following steps: determining at least one test flow model, a test data model and a model driving file corresponding to a protocol to be detected; traversing the spliced at least one test flow model to obtain at least one test path, and each test path has a corresponding test operation sequence; for each of at least part of the test paths, generating a test case based on the test operation sequence and test data corresponding to the test path; wherein the test data comprises data determined from the test data model based on the protocol to be detected and the model driving file, and the test data serves as a test parameter value of a test operation in the test operation sequence. The application can improve test convenience on the basis of improving test accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of software testing, and particularly relates to a method and device for generating test cases and an electronic device. BACKGROUND

[0002] The security design and implementation of network protocols are not only related to people's privacy and property security, but also related to the interests of the country. Network protocol vulnerabilities have become a research hotspot in the field of information security.

[0003] The applicant finds that in the field of industrial control (industrial control, for short) security control, control and testing of numerous industrial protocols are involved. The testing of industrial control protocols needs to cover dozens of protocols, each protocol has different characteristics, and the testing dimensions concerned in each testing process are uncertain. Therefore, when performing protocol testing each time, the mutual influence between each testing dimension and different testing dimensions may need to be fully tested, and a large amount of human and time costs are needed to summarize and supplement each protocol one by one to obtain applicable test cases. SUMMARY

[0004] To at least partially solve the problems in the related art, the present application provides a method and device for generating test cases and an electronic device, which can effectively improve the convenience of obtaining test cases and the coverage rate of test cases.

[0005] A first aspect of the present application provides a method for generating test cases, comprising: determining at least one test flow model, a test data model and a model driving file corresponding to a to-be-detected protocol; traversing the spliced at least one test flow model to obtain at least one test path, each test path having a corresponding test operation sequence; for each of at least part of the test paths, generating a test case based on the test operation sequence and test data corresponding to the test path; wherein the test data comprises data determined from the test data model based on the to-be-detected protocol and the model driving file, and the test data is used as a test parameter value of a test operation in the test operation sequence.

[0006] The second aspect of the present application provides a device for generating test cases, comprising a model determination module, a model traversal module and a test case generation module. The model determination module is configured to determine at least one test flow model, a test data model and a model driving file corresponding to a to-be-detected protocol. The model traversal module is configured to traverse the spliced at least one test flow model to obtain at least one test path, and each test path has a corresponding test operation sequence. The test case generation module is configured to, for each of at least part of the test paths, generate a test case based on the test operation sequence and the test data corresponding to the test path. The test data comprises data determined from the test data model based on the to-be-detected protocol and the model driving file, and the test data is used as a test parameter value of a test operation in the test operation sequence.

[0007] The third aspect of the present application provides an electronic device, comprising a processor, and a memory having executable code stored thereon, which, when executed by the processor, causes the processor to perform the above method.

[0008] The fourth aspect of the present application further provides a computer-readable storage medium having executable code stored thereon, which, when executed by a processor of an electronic device, causes the processor to perform the above method.

[0009] The fifth aspect of the present application further provides a computer program product comprising executable code, which, when executed by a processor, implements the above method.

[0010] The method, device and electronic device for generating test cases provided by the present application can pre-construct test flow models and test data models for different test scenarios. These test flow models and test data models can be protocols that have been verified multiple times and can be reused. When testing a test protocol, these test flow models and test data models can be directly called for testing. Different test dimensions and the mutual influence between different test dimensions are comprehensively considered, and a large amount of manpower and time costs are not required, thereby effectively improving the test effect.

[0011] In addition, in some embodiments, the test data can be a field from the protocol content of the industrial control protocol, so as to determine which boundary conditions need to be tested, to improve the coverage of boundary conditions in the test process, and to cope with test scenarios where test traffic data is insufficient.

[0012] In addition, in some embodiments, the test data can be adjusted according to the target test parameter selected by the user, and only the test operation related to the target test parameter is tested, thereby effectively reducing the redundancy of the test parameters, improving the resource utilization efficiency and improving the test efficiency.

[0013] In addition, in some embodiments, a plurality of test paths can be determined in a traversal manner, and a test operation sequence of at least part of the test paths is determined, thereby improving the flexibility of the test and meeting various user test requirements.

[0014] It should be understood that the general description above and the following detailed description are only exemplary and explanatory, and cannot limit the present application. BRIEF DESCRIPTION OF DRAWINGS

[0015] The present application and other objects, features and advantages will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings, in which:

[0016] Figure 1 An exemplary system architecture to which the method, device and electronic equipment for generating test cases according to embodiments of the present application can be applied is schematically shown;

[0017] Figure 2 A flowchart of a method for generating test cases according to embodiments of the present application is schematically shown;

[0018] Figure 3 A schematic diagram of a commonly used protocol of an industrial control network according to embodiments of the present application is schematically shown;

[0019] Figure 4 A structural schematic diagram of a test flow model according to embodiments of the present application is schematically shown;

[0020] Figure 5 A process schematic diagram of a detection protocol according to embodiments of the present application is schematically shown;

[0021] Figure 6 A structural schematic diagram of a spliced test flow model according to embodiments of the present application is schematically shown;

[0022] Figure 7 A structural schematic diagram of a test flow model before adjustment according to embodiments of the present application is schematically shown;

[0023] Figure 8 A schematic diagram of an adjusted test flow model according to embodiments of the present application is schematically shown;

[0024] Figure 9 A flowchart of a method for testing based on test cases according to embodiments of the present application is schematically shown;

[0025] Figure 10 A block diagram of a device for generating test cases according to embodiments of the present application is schematically shown;

[0026] Figure 11 A block diagram of an electronic device according to an embodiment of the present application is schematically shown. DETAILED DESCRIPTION

[0027] Embodiments of the present application will be described in more detail with reference to the accompanying drawings. While the embodiments of the present application are shown in the drawings, it is understood that the present application can be embodied in many different forms and should not be construed as being limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the application to those skilled in the art.

[0028] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the present application. As used herein, the term "includes" and tautological expressions thereof, such as "including," "includes," "include," "contains," "containing," and so forth, mean the term "comprises," "comprising," "comprises," "comprising," "comprises," "comprising," and so forth.

[0029] All terms used herein, including technical and scientific terms, have the meanings commonly understood by one of ordinary skill in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having meanings consistent with the context of the specification, and should not be interpreted in an idealized or overly formal manner.

[0030] It should be understood that although the terms "first," "second," "third," etc. can be employed in this application to describe various information, these information should not be limited to these terms. These terms are only used to differentiate one piece of information from another piece of information. For example, the first information can also be called the second information without departing from the scope of the present application, and similarly, the second information can also be called the first information. Therefore, the features defined as "first," "second" can explicitly or implicitly include one or more of the features. In the description of the present application, the meaning of "a plurality of" is two or more, unless otherwise specifically limited.

[0031] In order to facilitate the understanding of the technical solutions of the present application, first, some terms are explained.

[0032] Industrial control system is the combination of Ethernet and control network, which is born from the requirement of large data volume and high speed transmission of image, voice signal and the like. The networkization trend of industrial control system integrates many popular technologies such as multi-standard industrial control network interconnection and wireless technology, which expands the development space of industrial control field.

[0033] Network protocol is a set of communication rules between network nodes, which specifies the format that information must adopt when communicating and the meaning of these formats. Network nodes include, but are not limited to, at least one of a network server, a switch, and a router.

[0034] Finite state machine, a mathematical model representing a finite number of states and behaviors such as transitions between states and actions.

[0035] The related art needs to cover dozens of protocols when performing protocol testing. In order to improve the testing efficiency, the configuration and functions of various industrial protocols of industrial control can be tested by using an automatic script testing method. However, each protocol has different characteristics, and the testing dimensions are uncertain. Therefore, each dimension and the mutual influence between different dimensions need to be fully tested, which requires a large amount of manpower and time costs to summarize and supplement each protocol in turn. In addition, based on the application scenario of industrial protocols, the test traffic data obtained is limited, and deeper testing needs to be performed for this scenario, such as easily missing the testing of boundary conditions.

[0036] After the applicant summarizes the related art, it is found that the related art at least has the following defects when performing protocol testing.

[0037] For example, with the increase of industrial control protocols, the accumulation of corresponding scripts is accelerated, resulting in an increase in maintenance costs.

[0038] For example, the message structure in the actual network environment is various, and the testing depth and breadth are limited by the experience of the tester, and the testing of the protocol is uncertain. For example, there are various bus networks in the field bus system which has a great influence in the control field. Although the industry has formulated a unified standard (FF), due to commercial profits, technical monopolies and other reasons, the field bus products are still in a situation of hundred flowers blooming, resulting in a higher requirement for the experience of the tester.

[0039] For example, the testing method is difficult to reuse due to different characteristics of different protocols.

[0040] For example, the test data (such as test traffic messages) is limited, and the coverage of the test scene is less.

[0041] In the embodiments of the present application, test flow models and test data models and the like are constructed in advance for different types of protocols. These test flow models and test data models and the like can be models that have been verified multiple times and can be reused. When some protocols need to be tested, the test flow model and the test data model corresponding to the protocol can be directly called, and the mutual influence between different testing dimensions is comprehensively considered, without the need to consume a large amount of manpower and time costs to determine the mutual influence, thereby effectively improving the testing accuracy and convenience.

[0042] The following will be combined Figures 1 to 11 The technical solutions of the present application are exemplarily described.

[0043] Figure 1An exemplary system architecture to which the method, apparatus and electronic device for generating test cases according to embodiments of the present application can be applied is schematically shown. It should be noted that Figure 1 The shown are only examples of scenarios to which embodiments of the present application can be applied, to help those skilled in the art understand the technical content of the present application, but do not mean that embodiments of the present application cannot be applied to other devices, systems, environments or scenarios.

[0044] Referring to Figure 1 Taking the industrial control network 1 as an example, there are multiple devices 10, 20, 30, 40, 50, etc. in the factory, and the devices 10, 20, 30, 40, 50 can produce according to the scheduling information according to the preset process parameters, etc. In addition, there can be multiple sensors in the factory to monitor environmental information, such as monitoring temperature information through the temperature sensor 60 and monitoring humidity information through the humidity sensor 70. The devices 10, 20, 30, 40, 50, the temperature sensor 60, the humidity sensor 70, etc. can be connected to the control terminal 80 through the network 90. The information fed back by the devices 10, 20, 30, 40, 50, etc. includes production-related production factor information. For example, device status, temperature information, humidity information, etc. can be stored in the control terminal 80 and can be updated in real time, so as to facilitate the control terminal 80 to arrange production based on these real-time acquired production factors. In addition, the control terminal 80 can also be connected to the server 100 through the network 90 to receive or send information such as production factor information. It should be noted that the network 90 can include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0045] For example, the server 100 or the control terminal 80, etc. generates test cases in response to a test request, and can detect a specific protocol based on the test cases.

[0046] It should be understood that the number of devices, networks, control terminals and servers is only illustrative. According to the implementation needs, there can be any number of edge ends, networks and servers.

[0047] Figure 2 A flowchart of a method for generating test cases according to an embodiment of the present application is schematically shown.

[0048] Referring to Figure 2 The embodiment provides a method for generating test cases, which includes operations S210-S230, as follows.

[0049] In operation S210, at least one test flow model, test data model and model driving file corresponding to the protocol to be detected are determined.

[0050] In the embodiment, the user can specify the protocol to be detected, or the system can automatically screen the protocol to be detected. The protocol to be detected can be various communication protocols such as industrial control protocols.

[0051] Figure 3 The schematic diagram of the commonly used protocol of the industrial control network according to the embodiment of the application is shown schematically. Referring to Figure 3 The protocol to be detected can be a traditional control network protocol, a fieldbus protocol, an industrial Ethernet protocol, an industrial wireless network protocol, etc.

[0052] The traditional control network protocol includes but is not limited to a centralized digital control system (CCS), a distributed control system (DCS), a fieldbus control system (FCS), a reference link, etc.

[0053] The fieldbus protocol includes but is not limited to a controller area network (CAN), a device network, a control and communication link (CC-Link), a process field bus (PROFIBUS), etc.

[0054] The industrial Ethernet protocol includes but is not limited to an Ethernet / industrial protocol (Ethernet / IP), an Ethernet for control automation technology (EtherCAT), a high-speed Ethernet (HSE), a Profinet, an Ethernet for plant automation (EPA), a POWERLINK, a POWERLNK, a Modbus, a neighbor discovery protocol 3 (NDP3), an OLE for process control (OPC), a Tcnet developed by Toshiba, a Wnet, etc.

[0055] Industrial wireless network protocols include, but are not limited to, IEEE 802 standards developed by the Local Area Network / Metropolitan Area Network Standards Committee (LAN / MAN Standards Committee), R Fieldbus, ZigBee, etc.

[0056] The development trend of the protocol of the control network is gradually tending to open and transparent communication protocol. Ethernet has the advantages of high transmission speed, low consumption, easy installation and good compatibility. Since it supports almost all popular network protocols, it is widely used in commercial systems. With the development of network technology, Ethernet has entered the control field, forming the control network technology based on Ethernet. This is at least partly due to the development of industrial automation systems towards distributed and intelligent control, and open and transparent communication protocol is a possible trend. Fieldbus cannot meet this requirement due to its variety and incompatibility. However, the TCP / IP protocol of Ethernet and other protocols cannot replace the fieldbus protocol in the short term, and both will coexist for a long time, which further increases the types of protocols that need to be tested and the difficulty of testing.

[0057] In this embodiment, the test flow model and the test data model for different types of protocols can be abstracted from historical test cases based on experience, rules, etc. These test flow models and test data models can be reused to ensure the accuracy and coverage of each model and file. When a certain type of protocol needs to be tested, the user inputs the protocol to be tested, and the server can automatically select the test flow model and the test data model corresponding to the protocol to be tested. The user can also manually select the test flow model and the test data model corresponding to the protocol to be tested. The model-driven file can also be a pre-edited file, and the user can also edit the driving model file according to his own needs.

[0058] Specifically, determining the at least one test flow model, the test data model and the model-driven file corresponding to the protocol to be tested can include the following operations: determining the at least one test flow model and the test data model corresponding to the protocol to be tested based on the mapping relationship in the model-driven file. The protocol to be tested includes test parameters, the mapping relationship includes the correspondence between the test parameters, the test flow model and the test data model, the test flow model includes at least one node and at least one edge, the node represents a state machine model, the edge represents a test operation, and the test operation is executed to jump between different state machine models.

[0059] For a network protocol including multiple states, it is a protocol related to context information, data information and historical track. The communication process of the network protocol including multiple states is very complex. For example, it can include processes such as handshake and authentication. When the related technology performs protocol testing, the test data used can only cover the first interaction state, and it is difficult to cover the subsequent state track. Therefore, the network protocol including multiple states needs to consider the state characteristics of the stateful network protocol.

[0060] The state machine model is suitable for describing the state transition characteristics of the network protocol, and is a form of formal description that can be used. The state machine model is generally represented by a directed graph, the vertex represents the state, and the directed edge represents the operation, and the state transition can be realized through the operation. The input and output marked on the directed edge are the conditions for state transition. The state machine model not only has strong intuitiveness, but also is convenient for automatic implementation. For example, the state machine model can be represented as a multi-tuple. The state machine model can be a finite state machine model.

[0061] For example, the state machine model can include multiple states as shown in the following: a read coil state, a coil state in state 1, a coil state in state 2, and the like. The test operation can be receiving message 1, and the corresponding state transition is that the coil state migrates from state 1 to state 2.

[0062] In operation S220, at least one test path is obtained by traversing the spliced at least one test flow model, and each test path has a corresponding test operation sequence.

[0063] In this embodiment, since multiple test flow models and / or test data models can be involved in the process of testing a protocol, the multiple test flow models need to be spliced to obtain a complete test model for traversal in the process of generating a test case.

[0064] For example, the test flow model can include multiple state machine models and test operations, and the jump between multiple state machines can be realized through different test operations, which facilitates the judgment of whether the test result is correct based on the state.

[0065] Figure 4 The structure of the test flow model according to the embodiment of the application is schematically shown.

[0066] Referring to Figure 4The test flow model can include state machine model 1 to state machine model 7, which can be displayed as nodes, and there can be a connection relationship between the nodes. The connection relationship can represent a test operation, which realizes the jump between different state machines. At least part of the test flow model includes an entrance and an exit, so as to connect different test flow models. In some embodiments, at least part of the test flow model includes an entrance and an exit.

[0067] Reference Figure 4 As shown, the test flow model can include three test paths.

[0068] For example, test path 1: state machine model 1-state machine model 2-state machine model 6-state machine model 7. Correspondingly, the test operation sequence can include: test operation 0-test operation 1-test operation 2-test operation 6-test operation 7. Each test operation can be the same or different.

[0069] For example, test path 2: state machine model 1-state machine model 2-state machine model 3-state machine model 4-state machine model 6-state machine model 7. Correspondingly, the test operation sequence can include: test operation 0-test operation 1-test operation 2-test operation 3-test operation 4-test operation 6-test operation 7. Each test operation can be the same or different.

[0070] For example, test path 3: state machine model 1-state machine model 2-state machine model 3-state machine model 4-state machine model 5-state machine model 6-state machine model 7. Correspondingly, the test operation sequence can include: test operation 0-test operation 1-test operation 2-test operation 3-test operation 4-test operation 5-test operation 6-test operation 7. Each test operation can be the same or different.

[0071] It should be understood that when multiple test flow models are spliced, more test paths can be obtained.

[0072] In operation S230, for each of at least part of the test paths, a test case is generated based on the test operation sequence and the test data corresponding to the test path.

[0073] The test data includes data determined from the test data model based on the protocol to be detected and the model-driven file, and the test data is used as a test parameter value of the test operation in the test operation sequence.

[0074] The test operation can include at least one parameter, and the test data can assign a value to the at least one parameter.

[0075] In some embodiments, generating the test case based on the test operation sequence and the test data corresponding to the test path can include the following operation: for each test operation in the test operation sequence, assigning the test data to the test parameter corresponding to the test operation, and generating an executable test operation.

[0076] The model driving file is used as a parameter of model instantiation to drive the operation of the whole model. For the industrial control protocol, the test object and test range of each industrial control protocol are defined in the model driving file. In addition, the model driving file can also specify the validity of the test data.

[0077] Figure 5 The process of detecting the protocol according to the embodiments of the present application is schematically shown.

[0078] Referring to Figure 5 The dashed box represents the test model related data, which can be data called from a pre-developed library. For example, a database including a test flow model and a database including a test data model can be developed. In addition, a database including a driving file can also be developed. When a user needs to test a certain type of protocol, at least one of the corresponding test flow model, test data model and model driving file can be selected from the library to generate a test case, and the test is performed based on the test case. The test flow model can be obtained by flow modeling. The test data model can be constructed based on historical test data. It should be noted that the user can configure the model driving file and the like according to the needs, such as at least one of adding, deleting or modifying the setting information.

[0079] For example, the model driving file can provide protocol related constraints, such as conditions 1 to 4.

[0080] Condition 1, log format of the protocol record.

[0081] Condition 2, number of protocol feature parameters to be tested.

[0082] Condition 3, value range of the protocol feature parameter to be tested.

[0083] Condition 4, mapping relationship of the protocol feature parameter to be tested.

[0084] The above constraints are only exemplary and should not be construed as limiting the present application.

[0085] For example, in the embodiment, a model-based testing method can be used to model the entire testing process of the industrial control protocol, construct a test data model and a test process model. Then, the test data model and the test process model can be instantiated to generate test cases by taking the model-driven file as input. In this way, the test cases can be executed, and the detection of the protocol can be based on the check results.

[0086] In some embodiments, the above method can further include the operation of constructing a test data model. At least part of the test data in the test data model of the industrial control protocol is from the fields of the protocol content of the industrial control protocol. The fields selected from the protocol content are used as test data, so that the embodiment can be applied to the scene with limited industrial traffic for coverage testing.

[0087] Specifically, the test data model can include at least one of testable fields of the protocol, value ranges corresponding to the testable fields, conditions matched with message features, and constraint conditions.

[0088] For example, the test data of the industrial control protocol can be derived from various fields of the protocol content. For traffic with certain characteristics, the industrial control device (such as an industrial control firewall) can correctly identify the behavior of the traffic and take corresponding actions.

[0089] In some embodiments, constructing the test data model can include the following operations. First, test sample data is obtained, and the test sample data includes test parameter values. Then, at least part of the dimensions of the test parameter values and the combinations of the parameter values are processed by reverse matching to obtain a plurality of test data entries. Next, the plurality of test data entries are used as the test data model.

[0090] Specifically, the actual test environment does not cover all scenarios, and a reverse matching test method can be used to reduce the amount of redundant testing. For example, by modeling the known industrial protocol traffic message based on message description, using pairing algorithms (such as Pairwise / All-Pairs Testing) or orthogonal method testing strategies (Orthogonal Array Testing Strategy, referred to as OATS), etc., all dimensions and value combinations of test variables of a certain protocol are screened, and exhaustive testing of all dimensions and their combinations is avoided to obtain effective test data, and various possibilities of the parsing engine are tested.

[0091] For example, the test parameters include categories, device groups, and product test results. The categories include category 1 and category 2. The device groups include group 1 and group 2. The product test results include qualified and unqualified. After permutation and combination, 8 groups of test data can be obtained, as shown in Table 1.

[0092] Table 1

[0093] Serial number Class Device group Product test result 1 Class 1 Group 1 Pass 2 Class 2 Group 1 Pass 3 Class 1 Group 2 Pass 4 Class 2 Group 2 Pass 5 Class 1 Group 1 Fail 6 Class 2 Group 1 Fail 7 Class 1 Group 2 Fail 8 Class 2 Group 2 Fail

[0094] The reverse matching process can be as follows.

[0095] First, analyze from serial number 8. Serial number 8 is a combination of "category 2, group 2, unqualified", and the pairwise combination is "category 2 group 2", "group 2 unqualified", "category 2 unqualified". Check if these three combinations have appeared in serial numbers 1-7, it can be seen that "category 2 group 2" appears in 4, "group 2 unqualified" appears in 7, and "category 2 unqualified" appears in 6. Therefore, according to the idea of paired test method, 8 can be discarded. At this time, the remaining test data is shown in Table 2.

[0096] Table 2

[0097]

[0098]

[0099] Next, analyze serial number 7. The pairwise combination of serial number 7, "category 1 group 2", appears in 3, "category 1 unqualified" appears in serial number 5, but "group 2 unqualified" is only this one, so serial number 7 needs to be retained.

[0100] Next, analyze serial number 6. By the same reasoning, it can be obtained that the combination of serial number 6, "category 2 group 1", appears in 2, "group 1 unqualified" appears in 5, and "category 2 unqualified" is only this one, so 6 needs to be retained.

[0101] Next, analyze serial number 5. By the same reasoning, it can be obtained that the combination of 5, "category 1 group 1", appears in 1, "group 1 unqualified" appears in 6, and "category 1 unqualified" appears in 7, so 6 can be discarded. The remaining test data is shown in Table 3.

[0102] Table 3

[0103] Serial number Class Device group Product test result 1 Class 1 Group 1 Pass 2 Class 2 Group 1 Pass 3 Class 1 Group 2 Pass 4 Class 2 Group 2 Pass 6 Class 2 Group 1 Fail 7 Class 1 Group 2 Fail

[0104] Next, analyze serial numbers 4, 3, 2, and 1 in turn in the above manner. The final retained test data is shown in Table 4.

[0105] Table 4

[0106]

[0107]

[0108] It can be seen that the test data after the reverse matching processing is half of the original, effectively reducing the test data, and having good test coverage. It should be noted that the above reverse matching processing is only exemplary and cannot be construed as limiting the present application.

[0109] The test data model specifies the testable fields and value ranges of the protocol, the conditions matched with the message characteristics, and some general constraints, so that the test data is more targeted.

[0110] For example, the test data can be part of the test data obtained by the pariwise algorithm, and the test data covers multiple groups of test data for forward and reverse testing. The test data can be as shown in Table 5.

[0111] Table 5

[0112] arg1 arg2 arg3 match 001 Read Coil (read coil status) -1 0 N Any -1 100 Y 015 Write Multiple Coils (force multiple coils) 0 65535 N 004 Read Input Registers (read input registers) 0 0 N 024 Read FIFO Queue 65535 65535 N 003 Read Holding Registers (read holding registers) 1 100 N 006 Write Single Register (preparation single register) 65535 65535 N Any 0 0 N 016 Write Multiple Registers (preparation multiple registers) 0 0 N 006 Write Single Register (preparation single register) 1 100 N 002 Read Discrete Inputs (read input status) -1 65535 N 023 Read Write Register -1 0 N 001 Read Coil (read coil status) 1 65535 Y 005 Write Single Coil (force single coil) -1 -1 N 022 Mask Write Register 1 65535 N

[0113] In some embodiments, the above method can further include constructing a test flow model.

[0114] The control test of the industrial protocol can include two parts, one part is to configure the strategy related to the industrial protocol (such as the characteristics related to the protocol or the action of control). The other part is the actual control check of the flow. In the previous embodiment, the test data has been obtained, and the process of constructing the test flow model is to obtain the test operation by modeling, so as to combine into a complete test case.

[0115] The modeling of the test step first analyzes the function of the module, and then analyzes the relationship between the test steps and the state machine model. According to the state of the system under test, the constraint condition set in advance, the test design is made, and the test case is generated and executed. In addition, when generating the test case, the test strategy (see Figure 7 and Figure 8 related part) can also be used.

[0116] Specifically, constructing the test flow model can include the following operations.

[0117] First, draw the nodes corresponding to the state machine model, and determine the associated nodes of the nodes, and the state machine model is used to perform state transition according to the pre-set state. Then, connect the nodes and the associated nodes to obtain the edges. Next, a mapping relationship between the edges and the test operations is established to obtain the test flow model.

[0118] For example, using a drawing tool (such as graphwalker) to draw the state machine model and the connection relationship, describing each operation step of the test, connecting the operation steps that can possibly be connected, and drawing a complete model that covers all the processes as much as possible.

[0119] In one embodiment, the graphwalker is used to draw nodes and edges. Each edge represents a transition operation and each node represents a state. It should be noted that the multi-model combination needs to provide an exit and an entrance in each model to facilitate switching between models.

[0120] Figure 6 An illustrative diagram of a structure of a spliced test flow model according to an embodiment of the present application is shown.

[0121] Referring to Figure 6 , test flow model 1, test flow model 2 and test flow model 3 are shown. Test flow model 1 includes state machine models 1-7, test flow model 2 includes state machine models 8-10, and test flow model 3 includes state machine models 11-13. The state machine models 1-7 can be the same as or different from the state machine models 8-10.

[0122] In some embodiments, test cases can be generated in an automatic manner or in a manual manner.

[0123] Specifically, traversing the spliced at least one test flow model can include at least one of the following.

[0124] For example, in response to a received operation instruction, the at least one test flow model is spliced, and then the spliced at least one test flow model is traversed. The operation instruction is an instruction for the test flow model.

[0125] For example, the at least one test flow model is spliced according to a dependency relationship between respective functions of the at least one test flow model, and then the spliced at least one test flow model is traversed.

[0126] Figure 6 A simple test case generation process is described in the following. First, configuration is made, and after the model is generated, the model is loaded in a manual or automatic manner to generate a sequence of test steps. The finite state machine model is used in the example shown in the figure, and test cases are generated according to the characteristics of the state machine and the path traversal algorithm.

[0127] The traversal process can use any one or more of the following.

[0128] Finite state machine (FSM), automatic generation of test cases: different algorithms and strategies are combined to traverse the model.

[0129] The model lists all transitions and states according to a large number of different paths to automatically generate a series of test cases.

[0130] The main efficacy of the model-based testing technique is embodied in the path traversal algorithm. The traversal algorithm: random traversal, weighted traversal, cover all transitions, etc.

[0131] In some embodiments, splicing the at least one test flow model according to the dependency between the respective functions of the at least one test flow model can include connecting the entry of a test flow model with the exit of a dependent test flow model according to the dependency between the respective functions of the at least one test flow model.

[0132] For example, traversing the spliced at least one test flow model can include the following operations.

[0133] First, take the node corresponding to the entry of the first test flow model as the starting node, and then perform a depth-first traversal of all nodes from the unvisited connected nodes of the starting node in turn until the node corresponding to the exit of the last test flow model is visited.

[0134] Then, repeat the following operations until all nodes of all test flow models are visited: if there are unvisited nodes, select one of the unvisited nodes as the starting node, and then perform a depth-first traversal of the unvisited nodes from the unvisited connected nodes of the starting node in turn until the node corresponding to the exit of the last test flow model is visited.

[0135] For example, referring to Figure 6 , the test path can include: state machine model 1 - state machine model 2 - state state machine model 7 - state machine model 8 - state machine model 10 - state machine model 11 - state machine model 12 - state machine model 13. Other test paths are not shown one by one.

[0136] An industrial control protocol can support different operations on different functions and components of multiple industrial devices. In this embodiment, the needs of multiple test scenarios can be met by splicing multiple test flow models.

[0137] In some embodiments, the test flow model and / or the test data model can be adjusted for test parameters of interest to the user to improve test efficiency.

[0138] Specifically, the method can further include the following operations. First, in response to a target test parameter determined from the test parameters, the test flow model is adjusted to obtain a sub-test flow model, and target test data corresponding to the target test parameter is selected from the test data model. Then, the test flow model is replaced with the sub-test flow model. The target test parameter can be a parameter specified by a user. Since there is a corresponding relationship between the test parameters and the test data, the required test data can be determined based on the user-specified parameter.

[0139] For example, adjusting the test flow model to obtain the sub-test flow model can include the following operations. First, a state machine associated with the target test parameter is determined. Then, non-associated state machines and non-associated test operations outside the associated state machine are deleted from the test flow model to obtain the sub-test flow model.

[0140] Figure 7 An illustrative diagram of a structure of a test flow model before adjustment according to an embodiment of the present application is shown. Figure 8 An illustrative diagram of a test flow model after adjustment according to an embodiment of the present application is shown.

[0141] Referring to Figure 7 , a certain test flow model includes state machine model 1 to state machine model 12, wherein state machine model 1 is connected to state machine model 2, state machine model 6, state machine model 7, state machine model 9, state machine model 10, state machine model 11, and state machine model 12 respectively. Currently, test parameter A needs to be tested, and the state machine models associated with test parameter A include state machine model 6, state machine model 7, state machine model 9, state machine model 10, state machine model 11, and state machine model 12. Therefore, the test flow model shown in Figure 7 can be simplified to the test flow model shown in Figure 8 .

[0142] This model design method has the characteristics of dynamic self-adaptation. According to different protocol types or different test requirements in the model driving file, the nodes or processes of the model can be deleted and modified to automatically generate a model suitable for the current test requirement

[0143] Figure 9 An illustrative flowchart of a method for testing based on test cases according to an embodiment of the present application is shown. The test sample data can further include correct test result identification information.

[0144] Referring to Figure 9 , after operation S230 is performed, the method can further include operation S940 and operation S950.

[0145] At operation S940, the protocol to be detected is tested based on the test case, and a test result for the test case is obtained.

[0146] At operation S950, the correct test result corresponding to the correct test result identification information is compared with the test result.

[0147] For example, the test can be performed based on the generated multiple test cases, and the test process can be as follows.

[0148] For example, user→selects industrial protocol A→industrial protocol feature configuration (matching)→industrial control flow test→checks control result.

[0149] For example, user→selects industrial protocol B→industrial protocol feature configuration (not matching)→industrial control flow test→checks control result.

[0150] For example, user→selects industrial protocol B→industrial protocol feature configuration (matching)→control action→industrial control flow test→checks control result.

[0151] In this embodiment, the test data model is obtained based on the modeling of the description of the industrial protocol flow, and the test data is obtained based on the test data model. The test flow model is generated based on the modeling of the test flow of the industrial control device, and the test step sequence is obtained based on the test flow model, so as to generate the executable test case. The test case generated in the above manner has the characteristics of dynamic generation, traceability, and no need to maintain solid test cases. In addition, the user can generate specific models through self-defined data driving and self-adaptive generation.

[0152] Another aspect of the present application also provides a device for generating a test case.

[0153] Figure 10 A block diagram of a device for generating a test case according to an embodiment of the present application is schematically shown.

[0154] Referring to Figure 10 The device 1000 for generating a test case can include a model determination module 1010, a model traversal module 1020, and a test case generation module 1030.

[0155] The model determination module 1010 is configured to determine at least one test flow model, a test data model, and a model driving file corresponding to the protocol to be detected.

[0156] The model traversal module 1020 is configured to traverse the spliced at least one test flow model to obtain at least one test path, and each test path has a corresponding test operation sequence.

[0157] The test case generation module 1030 is configured to generate a test case based on a test operation sequence and test data corresponding to each of at least part of the test paths. The test data includes data determined from a test data model based on the protocol to be detected and a model driving file, and the test data is used as a test parameter value of a test operation in the test operation sequence.

[0158] In some embodiments, the model determination module is specifically configured to determine at least one test flow model and a test data model corresponding to the protocol to be detected based on a mapping relationship in the model driving file, wherein the protocol to be detected includes a test parameter, the mapping relationship includes a correspondence relationship among the test parameter, the test flow model, and the test data model, the test flow model includes at least one node and at least one edge, the node represents a state machine model, and the edge represents a test operation, and the test operation is used to jump between different state machine models.

[0159] In some embodiments, the apparatus 1000 can further include a test data model construction module configured to construct a test data model, wherein at least part of the test data in the test data model for the industrial control protocol is from a field of a protocol content of the industrial control protocol.

[0160] In some embodiments, the test data model includes at least one of a testable field of the protocol, a value range corresponding to the testable field, a condition matched with a message feature, and a constraint condition.

[0161] In some embodiments, the test data model construction module includes a data acquisition unit, an inverse matching unit, and a model replacement unit.

[0162] The data acquisition unit is configured to acquire test sample data, and the test sample data includes a test parameter value.

[0163] The inverse matching unit is configured to perform inverse matching processing on at least part of dimensions of the test parameter value and a combination of the parameter values to obtain a plurality of test data entries.

[0164] The model replacement unit is configured to use the plurality of test data entries as a test data model.

[0165] In some embodiments, the test sample data further includes correct test result identification information. Accordingly, the apparatus 1000 can further include a test module and a comparison module.

[0166] The test module is configured to test the protocol to be detected based on the test case to obtain a test result for the test case.

[0167] The comparison module is configured to compare a correct test result corresponding to the correct test result identification information with the test result.

[0168] In some embodiments, the apparatus 1000 further comprises a data and parameter determining module and a model adjusting module.

[0169] The data and parameter determining module is configured to adjust the test flow model according to a target test parameter determined from the test parameters, to obtain a sub-test flow model, and to select target test data corresponding to the target test parameter from the test data model.

[0170] The model adjusting module is configured to replace the test flow model with the sub-test flow model.

[0171] In some embodiments, the data and parameter determining module is specifically configured to determine a state machine associated with the target test parameter, and to delete non-associated state machines and non-associated test operations outside the associated state machine from the test flow model, to obtain the sub-test flow model.

[0172] In some embodiments, the apparatus 1000 further comprises a test flow model constructing module.

[0173] The test flow model constructing module comprises a node drawing unit, an edge drawing unit and a mapping establishing unit.

[0174] The node drawing unit is configured to draw a node corresponding to a state machine model, and to determine an associated node of the node, the state machine model being configured to perform state transition according to a pre-set state.

[0175] The edge drawing unit is configured to connect the node and the associated node to obtain an edge.

[0176] The mapping establishing unit is configured to establish a mapping relationship between the edge and a test operation, to obtain the test flow model.

[0177] In some embodiments, the model traversing module comprises a first traversing unit and a second traversing unit.

[0178] The first traversing unit is configured to splice at least one test flow model in response to a received operation instruction, the operation instruction being an instruction for the test flow model, and to traverse the spliced at least one test flow model.

[0179] The second traversing unit is configured to splice at least one test flow model according to a dependency relationship between respective functions of the at least one test flow model, and to traverse the spliced at least one test flow model.

[0180] In some embodiments, at least part of the test flow models each comprises an entrance and an exit. Accordingly, the second traversing unit is specifically configured to connect the entrance of a test flow model and the exit of a depended test flow model according to the dependency relationship between respective functions of the at least one test flow model, to obtain a test model.

[0181] In some embodiments, the second traversing unit comprises a traversing subunit and a loop subunit.

[0182] The traversing subunit is configured to take the node corresponding to the entrance of the first test procedure model as a starting node, and perform depth-first traversal on all nodes from the unvisited connected nodes of the starting node in sequence until the node corresponding to the exit of the last test procedure model is reached.

[0183] The loop subunit is configured to repeat the following operation until all nodes of all test procedure models are visited: if there is an unvisited node, take one node from the unvisited nodes as a starting node, and perform depth-first traversal on the unvisited nodes from the unvisited connected nodes of the starting node in sequence until the node corresponding to the exit of the last test procedure model is reached.

[0184] In some embodiments, the test case generation module 1030 is specifically configured to, for each test operation in the test operation sequence, assign the test data to the test parameter corresponding to the test operation to generate an executable test operation.

[0185] The device for generating a test case, the test procedure model and / or the test data model provided by the embodiments of the present application have high adaptability and are flexible to change, have no fixed cases, and are more convenient to maintain. In addition, the coverage test range is wider, and hidden problems and design defects are more likely to be found. In addition, the device is suitable for scenarios with limited industrial traffic for coverage testing.

[0186] As to the device 1000 in the above embodiments, the specific manners in which various modules, units and subunits perform operations have been described in detail in the embodiments of the method, and will not be described in detail here.

[0187] Another aspect of the present application also provides an electronic device.

[0188] Figure 11 A block diagram of an electronic device according to an embodiment of the present application is schematically shown.

[0189] Referring to Figure 11 The electronic device 1100 includes a memory 1110 and a processor 1120.

[0190] The processor 1120 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gates or transistor logic, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can be any conventional processor.

[0191] The memory 1110 can include various types of storage units, such as a system memory, a read-only memory (ROM), and a permanent storage device. Among them, the ROM can store static data or instructions required by the processor 1120 or other modules of the computer. The permanent storage device can be a readable and writable storage device. The permanent storage device can be a non-volatile storage device that does not lose stored instructions and data even after the computer is powered off. In some embodiments, the permanent storage device uses a mass storage device (such as a magnetic or optical disk, a flash memory) as a permanent storage device. In some other embodiments, the permanent storage device can be a removable storage device (such as a floppy disk, an optical drive). The system memory can be a readable and writable storage device or a volatile readable and writable storage device, such as a dynamic random access memory. The system memory can store some or all instructions and data required by the processor during runtime. In addition, the memory 1110 can include a combination of any computer readable storage media, including various types of semiconductor storage chips (such as DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), magnetic disks and / or optical disks. In some embodiments, the memory 1110 can include a readable and / or writable removable storage device, such as a compact disc (CD), a read-only digital versatile disc (such as DVD-ROM, double-layer DVD-ROM), a read-only Blu-ray disc, an ultra-density optical disc, a flash memory card (such as an SD card, a min SD card, a Micro-SD card, etc.), a magnetic floppy disk, etc. The computer readable storage medium does not include a carrier wave and a transient electronic signal transmitted through wireless or wired transmission.

[0192] The memory 1110 stores executable code, which, when processed by the processor 1120, can cause the processor 1120 to perform part or all of the above-mentioned methods.

[0193] Furthermore, the method according to the present application can also be implemented as a computer program or a computer program product, which comprises computer program code instructions for executing some or all of the steps of the above-mentioned method according to the present application.

[0194] Alternatively, the present application can also be implemented as a computer readable storage medium (or a non-transitory machine readable storage medium or a machine readable storage medium) having stored thereon executable codes (or computer programs or computer instruction codes) which, when executed by a processor of an electronic device (or a server, etc.), cause the processor to perform some or all of the steps of the above-mentioned method according to the present application.

[0195] The above has described the embodiments of the present application, the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes are obvious to those skilled in the art without departing from the scope and spirit of the described embodiments. The selection of the terms used herein is intended to best explain the principles of the embodiments, practical application, or improvement to the technology in the market, or to enable other ordinary skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for generating test cases, characterized in that, include: Determine at least one test process model, test data model, and model driver file corresponding to the protocol to be tested; wherein the test process model and / or the test data model are: reusable models for different types of protocols abstracted from historical test cases based on experience and rules; By traversing at least one of the spliced ​​test process models, at least one test path is obtained, and each test path has a corresponding test operation sequence. For each of at least some of the test paths, generate test cases based on the test operation sequence and test data corresponding to that test path; The test data includes data determined from the test data model based on the protocol to be detected and the model driver file, and the test data serves as the test parameter values ​​for the test operations in the test operation sequence.

2. The method according to claim 1, characterized in that, The determination of at least one test process model, test data model, and model driver file corresponding to the protocol to be tested includes: Based on the mapping relationship in the model driver file, at least one test process model and test data model corresponding to the protocol to be tested are determined. The protocol to be tested includes test parameters, and the mapping relationship includes the correspondence between the test parameters, the test process model and the test data model. The test process model includes at least one node and at least one edge. The node represents a state machine model, and the edge represents a test operation. The test operation is executed to jump between different state machine models.

3. The method according to claim 2, characterized in that, The method further includes: constructing a test data model, wherein at least a portion of the test data in the test data model for the industrial control protocol comes from fields of the protocol content of the industrial control protocol.

4. The method according to claim 3, characterized in that, The test data model includes at least one of the following: testable fields of the protocol, value ranges corresponding to the testable fields, conditions that match message characteristics, and constraints.

5. The method according to claim 3, characterized in that, The construction of the test data model includes: Acquire test sample data, which includes test parameter values; Reverse matching is performed on at least some dimensions and combinations of parameter values ​​of the test parameters to obtain multiple test data entries; Multiple test data entries are used as the test data model.

6. The method according to claim 5, characterized in that, The test sample data also includes correct test result identification information; The method further includes: The protocol to be detected is tested based on the test cases, and test results are obtained for the test cases. Compare the correct test result corresponding to the correct test result identification information with the test result.

7. The method according to claim 2, characterized in that, The method further includes: In response to the target test parameters determined from the test parameters, the test process model is adjusted to obtain a sub-test process model, and target test data corresponding to the target test parameters is selected from the test data model; Replace the test process model with the sub-test process model.

8. The method according to claim 7, characterized in that, The adjustment of the test process model to obtain a sub-test process model includes: Determine the state machine associated with the target test parameters; The sub-test process model is obtained by removing the non-associated state machines and non-associated test operations other than the associated state machines from the test process model.

9. The method according to claim 2, characterized in that, The method further includes: constructing a test process model, including: Draw the nodes corresponding to the state machine model and determine the associated nodes of the node. The state machine model is used to perform state transitions according to a pre-defined state. Connect the node and the associated node to obtain an edge; Establish the mapping relationship between the edges and the test operations to obtain the test process model.

10. The method according to claim 9, characterized in that, The traversal of at least one of the concatenated test process models includes: In response to a received operation instruction, at least one of the test process models is concatenated, and then the concatenated at least one test process model is traversed, wherein the operation instruction is an instruction specific to the test process model; or According to the dependencies between the functions of at least one of the test process models, the at least one test process model is concatenated, and then the concatenated at least one test process model is traversed.

11. The method according to claim 10, characterized in that, At least some of the test process models described each include an inlet and an outlet; The step of assembling at least one of the test process models according to the dependencies between the functions of each of the at least one test process model includes: According to the dependencies between the functions of at least one of the test process models, the entry point of the test process model and the exit point of the dependent test process model are connected.

12. The method according to claim 11, characterized in that, The step of traversing at least one of the spliced ​​test process models includes: Starting from the node corresponding to the entry point of the first test process model, we traverse all nodes in a depth-first manner, starting from the unvisited connected nodes of the starting node, until we reach the node corresponding to the exit point of the last test process model. Repeat the following steps until all nodes of all test process models have been visited: If there are unvisited nodes, select one of the unvisited nodes as the starting node, and then traverse the unvisited nodes in a depth-first manner, starting from the unvisited connected nodes of the starting node, until the node corresponding to the exit of the last test process model is visited.

13. The method according to any one of claims 1 to 12, characterized in that, The generation of test cases based on the test operation sequence and test data corresponding to the test path includes: For each test operation in the test operation sequence, the test data is assigned to the test parameters corresponding to that test operation to generate an executable test operation.

14. An apparatus for generating test cases, characterized in that, include: The model determination module is used to determine at least one test process model, test data model, and model driver file corresponding to the protocol to be tested; wherein, the test process model and / or the test data model are: reusable models for different types of protocols abstracted from historical test cases based on experience and rules; The model traversal module is used to traverse at least one of the spliced ​​test process models to obtain at least one test path, and each test path has a corresponding test operation sequence. A test case generation module is used to generate test cases for each of at least some test paths, based on the test operation sequence and test data corresponding to that test path; wherein, the test data includes data determined from the test data model based on the protocol to be detected and the model driver file, and the test data serves as the test parameter values ​​of the test operations in the test operation sequence.

15. An electronic device, characterized in that, include: processor; as well as A memory having executable code stored thereon, which, when executed by the processor, causes the processor to perform the method according to any one of claims 1-13.

16. A computer-readable storage medium, characterized in that, It stores executable code that, when executed by a processor of an electronic device, causes the processor to perform the method as described in any one of claims 1-13.

Images