Software defined network controller performance automation test system and method

Abstract

The invention relates to a system and a method for automatically testing the performance of a software defined network controller. The system comprises a soft switch flow generation capability automatic measurement unit, a data plane network automatic construction unit, a given load flow triggering unit and a southbound protocol message information statistical analysis unit. Through a software-defined network simulation tool and a widely used software switch, the network controller can be conveniently tested based on a novel southbound protocol which evolves continuously, and the advancement and the universality of a test system are guaranteed.

Description

Technical Field

[0001] The present application relates to the field of computer network technology, and in particular to a system and method for automated performance testing of a software-defined network controller. Background Art

[0002] Software-Defined Networking (SDN), as a network architecture and technology, achieves programmability and flexibility of the network by separating the control plane from the data plane. In a software-defined network, the data plane implements the forwarding function of data packets according to specific policies, and the control plane implements centralized control and logical centralized management of the network by using a network controller. As the core component of the software-defined network, the network controller is responsible for managing and controlling the forwarding behavior of the data plane, and its performance directly affects the operating efficiency and stability of the network. Once the controller fails, it may directly lead to the network not working properly. Due to the differences in system architecture, implementation technology, and programming language, the performance of network controllers is also different. In particular, with the rapid growth of network traffic and the increase in network complexity, the requirements for controller performance are also increasing. From the user's perspective, the performance of the network controller is an important basis for selecting and evaluating the controller. From the developer's perspective, the performance of the network controller is an important indicator for research and development. Therefore, network controller performance testing is a very important task.

[0003] In the SDN architecture, the network controller collects network status information, including topology and device status, by communicating with the data plane devices. According to the established network policies such as routing, it generates corresponding flow table entries containing matching rules and processing actions for data packets, and sends the flow table to the SDN switch of the data plane through the southbound protocol (such as OpenFlow). The SDN switch matches and processes the data packets entering the network according to the received flow table entries. If the data packet successfully matches a flow table entry in the flow table, it is processed according to the action of the flow table entry (such as forwarding, discarding, etc.). When the switch receives a data packet that cannot match the flow table entry, or the action list in the matching flow table entry specifies the operation of forwarding the data packet to the controller (such as Output = CONTROLLER), it will encapsulate it as a Packet-In message and send it to the controller. The controller generates a Packet-Out message or a Flow-Mod message according to the network policy and sends it to the switch to guide the processing of the data packet. The Packet-Out message directly instructs the switch how to process a specific data packet, while the Flow-Mod message is used to modify the flow table of the switch to guide the subsequent data packet processing.

[0004] Among the network controller performance indicators, an important core key performance indicator is the response delay characteristics of Packet-Out messages or Flow-Mod messages under a given load (for example, 100,000 messages per second, or 100,000 pps). This needs to be evaluated and tested in detail.

[0005] Currently, there are three main methods for network controller performance testing: one is the testing method based on physical SDN switches; the second is the simulation testing method based on event-driven; and the third is the procedural testing method based on simulation tools.

[0006] The test method based on physical SDN switches uses physical SDN switches to build a data plane network to test the network controller. Since the rate of Packet-In messages generated by triggering a single physical SDN switch is limited (according to our test of a certain brand of SDN switch, it can only output thousands of Packet-In messages per second), multiple SDN switches are often required to meet the specified controller load requirements. This type of method has advantages such as authenticity, but it faces disadvantages such as high cost and cumbersome testing process.

[0007] The event-driven simulation test method mainly tests by constructing southbound protocol messages sent by the data plane network to the network controller. It is a traditional method for SDN controller performance testing. This type of method mainly uses CBench or its derivative tools for testing. After CBench is running, it simulates the SDN switch to establish a connection with the SDN controller. After receiving the request message from the SDN controller, CBench replies with some pre-set reply messages. CBench can also actively construct other types of southbound protocol messages and send them to the SDN controller. The advantages of the event-driven simulation test method are low resource overhead and ease of use. However, due to the continuous development and evolution of the SDN southbound protocol, the event-driven simulation test method faces deficiencies in the authenticity of the simulation scenario, limited support for the southbound protocol, flexibility of the test process, accuracy of the test results, and scalability of the test tool.

[0008] The procedural testing method based on simulation tools mainly uses widely used open source SDN network simulation tools (mainly Mininet) for testing. First, the tester uses the Mininet network simulation tool to start the corresponding number of software switches according to the topology information and network controller information pre-configured by the tester, and establishes corresponding links between them to build a running data plane network. Secondly, the packet capture tool should be configured to capture specific packets. Then, the tester needs to use other tools to generate packets that can trigger the software switch to send a Packet-in message to the SDN controller, and then send it to the software switch. Finally, the packet capture results are analyzed to find the time when the software switch sends the Packet-in message and the time when the SDN controller sends the Packet-out message or the Flow-Mod message, and the corresponding delay characteristics are analyzed and calculated. The advantages of this method are good authenticity, the software switch used by the network simulation tool runs the actual software-defined network protocol, and the southbound protocol support is good. However, there are shortcomings such as high requirements for the software-defined network technical level of the tester and complex, cumbersome and time-consuming testing process.

[0009] These existing test methods have many shortcomings, and it is difficult to test performance indicators such as delay characteristics under given load conditions. In general, there is currently a lack of efficient, accurate, fast and flexible test methods and tools for network controller performance. Summary of the invention

[0010] Based on this, it is necessary to provide a software-defined network controller performance automation testing system and method to address the above technical problems.

[0011] A software-defined network controller performance automatic testing system, the system comprising:

[0012] Automatic measurement unit for soft switch traffic generation capability, automatic data plane network construction unit, given load traffic triggering unit, and southbound protocol message information statistical analysis unit;

[0013] The automatic measurement unit for the traffic generation capability of the soft switch builds a simulated network including a software switch and two or more virtual hosts, establishes a connection with the network controller to be tested, and then sends a specific type of protocol message at a high speed through the virtual host, and obtains the sending rate of the message reported by the software switch through monitoring and statistics;

[0014] The data plane network automatic construction unit calculates the number of software switches according to the test load of the specified network controller to be tested, the port and address of the network controller, and the rate at which the software switch reports messages, and constructs a data plane network consisting of software switches and virtual hosts capable of starting the number of software switches based on a network simulation tool according to the number of software switches;

[0015] The given load flow triggering unit calculates the message reporting rate of each software switch according to the test load and the number of software switches corresponding to the data plane network, and sends a specific type of protocol message of the message reporting rate from each virtual host connected to the software switch to trigger the software switch to report the corresponding southbound message protocol according to the message reporting rate;

[0016] The southbound protocol message information statistical analysis unit performs southbound protocol message information statistical analysis by monitoring the virtual interface connecting the virtual host and the software switch.

[0017] In one of the embodiments, the soft switch traffic generation capability automatic measurement unit uses a network simulation tool to construct a simulation network consisting of a software switch and two or more virtual hosts connected;

[0018] After waiting for the network to connect to the network controller to be tested, start monitoring and recording the traffic of the virtual host;

[0019] Start network communication between virtual hosts and generate communication messages that can trigger southbound protocol messages;

[0020] After the preset running time, the traffic monitoring and recording of the virtual host is turned off, and a specific type of communication message is selected from the traffic record as a test message;

[0021] Enable traffic monitoring and recording of southbound protocol messages on the software switch, send test messages from the virtual host at a specific rate v1, and continue the test for a specified duration s;

[0022] Disable the traffic monitoring and recording of southbound protocol packets of the software switch, and calculate the southbound protocol packet sending rate v2 of the software switch within the test duration s;

[0023] If v2 is consistent with v1, then v1 is increased and v2 and v1 are counted again. Otherwise, v2 is set to the peak sending rate of the southbound protocol packets of the software switch.

[0024] In one embodiment, the data plane network automatic construction unit calculates the number of software switches according to the test load of the specified network controller to be tested, the port and address of the network controller, and the rate at which the software switch reports messages:

[0025]

[0026] Where n is the calculated number of software switches, α is the software switch load factor, and a value of 1 indicates that the required number of software switches is calculated based on the peak sending rate.

[0027] In one embodiment, the given load flow triggering unit obtains the peak sending rate v2 of the southbound protocol message of the software switch, calls the data plane network automatic construction unit interface, and obtains the current data plane network topology information:

[0028] Start traffic monitoring and recording of virtual hosts, and complete network communication between all virtual hosts based on the data plane network topology information;

[0029] After waiting for all virtual hosts to complete network communication, close the traffic monitoring and recording of the virtual hosts, select a specific type of protocol message from the traffic record for each software switch as a test message that triggers the southbound protocol message sending rate of the software switch, thereby forming a test message set containing n test messages;

[0030] According to the southbound protocol message sending rate allocation strategy, the southbound protocol message sending rate / message sending rate is set for each software switch / connected virtual host in the data plane network topology, so that the total southbound protocol message load rate v3 of the controller is the sum of the southbound protocol message sending rates of each software switch v3 = v s1 +v s2 +…+v sn (v si ≤v2,i=1,2,…,n);

[0031] The virtual hosts connected to different software switches obtain their corresponding test messages from the test message set and send test messages at a set rate to trigger each software switch to send southbound protocol messages at a set rate, thereby triggering the given load flow of the controller to be tested, and at the same time calling the southbound protocol message information statistical analysis unit interface to start monitoring and recording of southbound protocol traffic.

[0032] In one of the embodiments, when the southbound protocol message information statistical analysis unit intercepts a report message on a virtual interface connecting each software switch and a network controller, it determines whether a specific field of the message needs to be modified according to the message field information. If so, the report message is sent after modification, and the timestamp, protocol version, message type, source / destination address, message length and message key fields of the report message are analyzed at the same time to form a southbound protocol data report record, which is added to the southbound protocol data report record table;

[0033] When the southbound protocol message information statistical analysis unit monitors the downlink message on each virtual interface connecting the software switch and the network controller, it analyzes the timestamp, protocol version, message type, source / destination address, message length and message key fields of the downlink message, forms a southbound protocol data downlink record, and adds it to the southbound protocol data downlink record table; searches the southbound protocol data reporting record table, finds the southbound protocol data reporting record corresponding to the reporting message that triggers the downlink message, calculates and records the corresponding response delay and other information, and adds it to the response delay record table.

[0034] In one of the embodiments, it further includes: a display unit;

[0035] Used to call the southbound protocol message information statistics analysis unit interface, obtain the response delay from the response delay record table, and display the response delay change information in real time;

[0036] Dynamically calculate and display the statistical information of the mean, variance, and probability distribution of the response delay.

[0037] A method for automated performance testing of a software-defined network controller, the method comprising:

[0038] By building a simulated network including a software switch and more than two virtual hosts, after establishing a connection with the network controller to be tested, a specific type of protocol message is sent at high speed through the virtual host, and the sending rate of the message reported by the software switch is obtained by monitoring and statistics;

[0039] According to the test load of the specified network controller to be tested, the port and address of the network controller, and the rate at which the software switch reports messages, the number of software switches is calculated, and according to the number of software switches, a data plane network consisting of software switches and virtual hosts capable of starting the number of software switches is constructed based on a network simulation tool;

[0040] According to the test load and the number of software switches corresponding to the data plane network, the message reporting rate of each software switch is calculated, and a specific type of protocol message of the message reporting rate is sent from each virtual host connected to the software switch to trigger the software switch to report the corresponding southbound message protocol according to the message reporting rate;

[0041] By monitoring the virtual interface connecting the virtual host and the software switch, statistical analysis of southbound protocol message information is performed.

[0042] In one embodiment, it also includes:

[0043] Use network simulation tools to build a simulated network consisting of a software switch and two or more virtual hosts;

[0044] After waiting for the network to connect to the network controller to be tested, start monitoring and recording the traffic of the virtual host;

[0045] Start network communication between virtual hosts and generate communication messages that can trigger southbound protocol messages;

[0046] After the preset running time, the traffic monitoring and recording of the virtual host is turned off, and a specific type of communication message is selected from the traffic record as a test message;

[0047] Enable traffic monitoring and recording of southbound protocol messages on the software switch, send test messages from the virtual host at a specific rate v1, and continue the test for a specified duration s;

[0048] Disable the traffic monitoring and recording of southbound protocol packets of the software switch, and calculate the southbound protocol packet sending rate v2 of the software switch within the test duration s;

[0049] If v2 is consistent with v1, then v1 is increased and v2 and v1 are counted again. Otherwise, v2 is set to the peak sending rate of the southbound protocol packets of the software switch.

[0050] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0051] By building a simulated network including a software switch and more than two virtual hosts, after establishing a connection with the network controller to be tested, a specific type of protocol message is sent at high speed through the virtual host, and the sending rate of the message reported by the software switch is obtained by monitoring and statistics;

[0052] According to the test load of the specified network controller to be tested, the port and address of the network controller, and the rate at which the software switch reports messages, the number of software switches is calculated, and according to the number of software switches, a data plane network consisting of software switches and virtual hosts capable of starting the number of software switches is constructed based on a network simulation tool;

[0053] According to the test load and the number of software switches corresponding to the data plane network, the message reporting rate of each software switch is calculated, and a specific type of protocol message of the message reporting rate is sent from each virtual host connected to the software switch to trigger the software switch to report the corresponding southbound message protocol according to the message reporting rate;

[0054] By monitoring the virtual interface connecting the virtual host and the software switch, statistical analysis of southbound protocol message information is performed.

[0055] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0056] By building a simulated network including a software switch and more than two virtual hosts, after establishing a connection with the network controller to be tested, a specific type of protocol message is sent at high speed through the virtual host, and the sending rate of the message reported by the software switch is obtained by monitoring and statistics;

[0057] According to the test load of the specified network controller to be tested, the port and address of the network controller, and the rate at which the software switch reports messages, the number of software switches is calculated, and according to the number of software switches, a data plane network consisting of software switches and virtual hosts capable of starting the number of software switches is constructed based on a network simulation tool;

[0058] According to the test load and the number of software switches corresponding to the data plane network, the message reporting rate of each software switch is calculated, and a specific type of protocol message of the message reporting rate is sent from each virtual host connected to the software switch to trigger the software switch to report the corresponding southbound message protocol according to the message reporting rate;

[0059] By monitoring the virtual interface connecting the virtual host and the software switch, statistical analysis of southbound protocol message information is performed.

[0060] The above-mentioned software-defined network controller performance automatic testing system and method, first, the system includes a soft switch traffic generation capacity automatic measurement unit, a data plane network automatic construction unit, a given load traffic trigger unit and a southbound protocol message information statistical analysis unit, and a network simulation tool is used to build a simulated network including a software switch and more than two virtual hosts, so as to test the sending rate of the software switch report message, and then based on the test load, the number of software switches is calculated, and according to the number of software switches, a data plane network consisting of software switches and virtual hosts that can start the number of software switches is built based on the network simulation tool. For the test load and the number of software switches corresponding to the data plane network, the message reporting rate of each software switch is calculated, and a specific type of protocol message of the message reporting rate is sent from each virtual host connected to the software switch to trigger the software switch to report the corresponding southbound message protocol according to the message reporting rate, and finally, the southbound protocol message information is statistically analyzed by monitoring the virtual interface connected to the virtual host and the software switch. The present invention relies on software-defined network simulation tools and widely used software switches to facilitate the testing of network controllers based on the evolving and new southbound protocols, thereby ensuring the advancement and universality of the test system. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 It is a framework diagram of the network controller performance automation test system proposed by the present invention;

[0062] Figure 2 Schematic diagram of a unit for automatically measuring the flow generation capability of a soft switch according to an embodiment of the present invention;

[0063] Figure 3 Schematic diagram of a data plane network automatic construction unit in an embodiment of the present invention;

[0064] Figure 4 is a schematic diagram of a given load flow triggering unit in an embodiment of the present invention;

[0065] Figure 5 Schematic diagram of a southbound protocol message information statistical analysis unit in an embodiment of the present invention;

[0066] Figure 6 It is an overall flow chart of the automated testing method in an embodiment of the present invention;

[0067] Figure 7 is a flow chart of automatic measurement of traffic generation capability of a soft switch in an embodiment of the present invention;

[0068] Figure 8 is a flow chart of automatic construction of a data plane network in an embodiment of the present invention;

[0069] Fig. 9 is a flow chart of a given load test of a controller to be tested in an embodiment of the present invention;

[0070] Fig.10 It is a flowchart of southbound protocol message information analysis, recording and statistical presentation in an embodiment of the present invention;

[0071] Fig.11 It is a detailed flow chart of the automated testing method after integrating the aforementioned parts of the process in an embodiment of the present invention. DETAILED DESCRIPTION

[0072] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0073] In one embodiment, Figure 1 As shown, a software-defined network controller performance automation test system is provided, comprising the following steps: a soft switch traffic generation capability automatic measurement unit, a data plane network automatic construction unit, a given load traffic triggering unit, and a southbound protocol message information statistical analysis unit.

[0074] The automatic measurement unit for the traffic generation capability of the soft switch builds a simulated network including a software switch and two or more virtual hosts. After establishing a connection with the network controller to be tested, a specific type of protocol message is sent at high speed through the virtual host, and the sending rate of the message reported by the software switch is obtained through monitoring and statistics.

[0075] The data plane network automatic construction unit calculates the number of software switches according to the test load of the specified network controller to be tested, the port and address of the network controller, and the rate at which the software switch reports messages. Based on the number of software switches, the data plane network consisting of software switches and virtual hosts that can start the number of software switches is constructed based on the network simulation tool.

[0076] The given load traffic triggering unit calculates the message reporting rate of each software switch according to the test load and the number of software switches corresponding to the data plane network, and sends a specific type of protocol message of the message reporting rate from the virtual host connected to each software switch to trigger the software switch to report the corresponding southbound message protocol according to the message reporting rate.

[0077] The southbound protocol message information statistical analysis unit performs southbound protocol message information statistical analysis by monitoring the virtual interface connecting the virtual host and the software switch.

[0078] In the above-mentioned software-defined network controller performance automation test system, first, the system includes a soft switch traffic generation capacity automatic measurement unit, a data plane network automatic construction unit, a given load traffic trigger unit and a southbound protocol message information statistical analysis unit, and a network simulation tool is used to build a simulated network including a software switch and more than two virtual hosts, so as to test the sending rate of the software switch report message, and then based on the test load, the number of software switches is calculated, and according to the number of software switches, a data plane network consisting of software switches and virtual hosts that can start the number of software switches is built based on the network simulation tool. For the test load and the number of software switches corresponding to the data plane network, the message reporting rate of each software switch is calculated, and a specific type of protocol message of the message reporting rate is sent from each virtual host connected to the software switch to trigger the software switch to report the corresponding southbound message protocol according to the message reporting rate, and finally, the southbound protocol message information is statistically analyzed by monitoring the virtual interface connected to the virtual host and the software switch. The present invention facilitates the testing of network controllers based on the evolving and new southbound protocols by relying on software-defined network simulation tools and widely used software switches, and ensures the advancement and universality of the test system.

[0079] In one embodiment, the step of automatically measuring the traffic generation capability of the soft switch includes:

[0080] S1, after waiting for the network to be connected to the network controller to be tested, the traffic monitoring and recording of the virtual host begins.

[0081] S2, start network communication between virtual hosts, and generate communication messages that can trigger southbound protocol messages.

[0082] S3, after a preset running time, the traffic monitoring and recording of the virtual host is turned off, and a specific type of communication message is selected from the traffic record as a test message.

[0083] S4, start the traffic monitoring and recording of the southbound protocol message of the software switch, send the test message from the virtual host at a specific rate v1, and continue the test for a specified time s.

[0084] S5, turn off the traffic monitoring and recording of the southbound protocol messages of the software switch, and count the southbound protocol message sending rate v2 of the software switch within the test duration s.

[0085] S6: If v2 is consistent with v1, then v1 is increased and v2 and v1 are counted again; otherwise, v2 is set as the peak sending rate of the southbound protocol message of the software switch.

[0086] Specifically, the soft switch traffic generation capability automatic measurement unit tests the traffic sending peak rate of the southbound protocol message of a single software switch, generates the traffic generation capability characteristic information of a single software switch according to the analysis result, and provides the information to the data plane network automatic construction unit and the given load traffic trigger unit. Figure 2 The figure shows a schematic diagram of a soft switch traffic generation capability automatic measurement unit according to an embodiment of the present invention, which is mainly composed of a soft switch generation module, a test message capture module, and a soft switch traffic peak value test and analysis module.

[0087] The soft switch generation module uses the Mininet network simulation tool and the Open vSwitch software switch tool to generate a test network topology that is connected to the controller under test and contains only a single software switch (the switch connects virtual host h1 and virtual host h2).

[0088] The test message capture module is used to capture test messages that can trigger the software switch to send OpenFlow protocol messages such as Packet_in. Specifically, the module uses virtual host h1 and virtual host h2 in the test network topology to communicate, and uses the Tcpdump tool to capture the messages generated during the communication process at the port where the virtual host and the software switch are connected as test messages. These messages can trigger the default flow table of the software switch, thereby triggering the software switch to send Packet_in messages.

[0089] The soft switch traffic peak test and analysis module is used to test the peak transmission rate of Packet_in and other OpenFlow protocol messages of a single software switch in the current deployment platform environment. Specifically, the module uses the Tcpreplay replay tool to replay the captured test message at a certain rate v1 at the port where the virtual host h1 is connected to the software switch, and uses the Tcpdump tool to monitor and record all OpenFlow protocol messages at the port where the software switch is connected to the controller to be tested, and then calculates its transmission rate v2 after filtering out the corresponding Packet_in message. If v2 is consistent with v1, the test message sending rate is increased by the specified step size step_size, so that v1 = v1 + step_size; if v2 is less than v1, v2 is regarded as the peak limit of the Packet_in message sending rate of a single software switch in the current deployment platform environment. The above test process is repeated many times, and v2 is obtained for each test and the average value is calculated. Finally, the module uses the calculated average value as the characteristic information of the software switch traffic generation capability, which is used to provide it to the relevant modules of the remaining units mentioned later as input data.

[0090] In another embodiment, the step of calculating the required number of software switches includes:

[0091] According to the specified test load of the network controller to be tested, the port and address of the network controller, and the rate at which the software switch reports messages, the number of software switches is calculated as:

[0092]

[0093] Where n is the calculated number of software switches, α is the software switch load factor, and a value of 1 indicates that the required number of software switches is calculated based on the peak sending rate.

[0094] further, Figure 3 The figure shows a schematic diagram of a data plane network automatic construction unit according to an embodiment of the present invention, which only includes a topology automatic analysis and generation module.

[0095] The software switch generation module first obtains the software switch traffic generation capability characteristic information v2 and the user-given controller load flow information v3 from the soft switch traffic peak test and analysis module and the controller flow trigger module of the given load flow trigger unit mentioned later, and then substitutes v2 and v3 into the calculation formula The required number of software switches is obtained. Finally, the Mininet network simulation tool and the Open vSwitch software switch tool are used to generate a test topology containing the required number of software switches. Finally, this module provides the test network topology information to the relevant modules of the given load flow trigger unit mentioned later for load testing of the controller under test.

[0096] In one embodiment, the steps of triggering a given load flow include:

[0097] S1, start traffic monitoring and recording of virtual hosts, and complete network communication between all virtual hosts according to the data plane network topology information;

[0098] S2, after waiting for all virtual hosts to complete network communication, close the traffic monitoring and recording of the virtual hosts, select a specific type of protocol message from the traffic record for each software switch as a test message that triggers the southbound protocol message sending rate of the software switch, thereby forming a test message set containing n test messages;

[0099] S3, according to the southbound protocol message sending rate allocation strategy, set the southbound protocol message sending rate / message sending rate for each software switch / connected virtual host in the data plane network topology, so that the total southbound protocol message load rate v3 of the controller is the sum of the southbound protocol message sending rates of each software switch v3 = v s1 +v s2 +…+v sn (v si ≤v2,i=1,2,…,n);

[0100] S4, the virtual hosts connected to different software switches obtain their corresponding test messages from the test message set, and send test messages at a set rate to trigger each software switch to send southbound protocol messages at a set rate, thereby triggering the given load flow of the controller to be tested, and at the same time calling the southbound protocol message information statistical analysis unit interface to start monitoring and recording of southbound protocol traffic.

[0101] Furthermore, the given load traffic triggering unit first captures the test message that can trigger it to send the southbound protocol message for each switch in the data plane network, and then allocates the southbound protocol message sending rate for each software switch according to the traffic generation capability characteristics of the software switch and the customized allocation strategy, and finally triggers it to output the reporting message of the corresponding rate, so that the sum of their rates is equal to the controller given load traffic rate set by the user. Figure 4 The figure shows a schematic diagram of a given load flow trigger unit according to an embodiment of the present invention, which is mainly composed of a test message capture module and a flow trigger module of a controller under test. This unit interacts with the other three units and is responsible for load simulation of the controller under test.

[0102] The message capture module described here has the same function as the message capture module of the automatic measurement unit of the soft switch traffic generation capability described above, and both are used to capture test messages that trigger the software switch to send OpenFlow protocol messages such as Packet_in. The difference is that the message capture module described above only captures test messages for a single software switch to test the switch message sending rate peak; the message capture module described here captures test messages for all software switches in the test network topology to form a test message set, which is used to trigger the load traffic of the controller to be tested. In other words, in order to ensure the authenticity of the test results, the traffic used for testing should be based on the actual generated test topology.

[0103] The traffic triggering module of the controller to be tested obtains a set of test messages from the test message capture module of this unit, obtains the test network topology information from the topology automatic analysis and generation module of the data plane network automatic construction unit, obtains the software switch traffic generation capability characteristic information from the soft switch traffic generation capability automatic measurement unit, and starts message replay after completing the traffic triggering preparation work of the controller to be tested.

[0104] In one embodiment, the step of analyzing and recording the southbound protocol message information includes:

[0105] S1, when a report message is intercepted on the virtual interface connecting each software switch and the network controller, it is determined whether a specific field of the message needs to be modified according to the message field information. If so, the report message is sent after modification, and the timestamp, protocol version, message type, source / destination address, message length and key fields of the report message are analyzed to form a southbound protocol data report record, which is added to the southbound protocol data report record table;

[0106] S2, when the southbound protocol message information statistical analysis unit monitors the downlink message on each virtual interface connecting the software switch and the network controller, it analyzes the timestamp, protocol version, message type, source / destination address, message length and message key fields of the downlink message, forms a southbound protocol data downlink record, and adds it to the southbound protocol data downlink record table; searches the southbound protocol data reporting record table, finds the southbound protocol data reporting record corresponding to the reporting message that triggers the downlink message, calculates and records the corresponding response delay and other information, and adds it to the response delay record table.

[0107] further, Figure 5 The figure shows a schematic diagram of a southbound protocol message information statistics analysis unit according to an embodiment of the present invention. This unit is mainly composed of a flow monitoring and recording module, a flow record processing module, a statistics analysis and display module, and a flow record storage module, and is used for monitoring and recording of OpenFlow protocol messages, result analysis and display, and storage of test data.

[0108] When the flow monitoring and recording module receives the opening / closing flow monitoring and recording instruction of the flow trigger module of the controller to be tested, it will open / close the monitoring and recording of the Openflow protocol message at the ports connected to the controller to be tested by all software switches in the test topology. When monitoring the reported message, it is determined whether it is necessary to modify the specific fields of the message. If modification is required, the reported message is sent after modification, and the timestamp, protocol version, message type, source / destination address, message length, message key fields and other information of the reported message are analyzed to form the OpenFlow protocol data reporting record; when monitoring the sent message, the timestamp, protocol version, message type, source / destination address, message length, message key fields and other information of the sent message are analyzed to form the OpenFlow protocol data sending record. The relevant records will be formatted into a specific data structure and temporarily stored in the OpenFlow protocol data reporting / sending record table.

[0109] In one embodiment, the southbound protocol message information statistics display step includes:

[0110] Call the southbound protocol message information statistics analysis unit interface, obtain the response delay from the response delay record table, and display the response delay change information in real time; dynamically calculate the statistical information of the mean, variance, and probability distribution of the response delay and display it.

[0111] The traffic record processing, statistical analysis and display module reads the record information from the OpenFlow protocol data reporting / distribution record table of the traffic monitoring and recording module at fixed intervals, calculates the response delay after completing the matching of the reporting / distribution records, processes and forms the response delay record, and adds it to the response delay record table. Subsequently, the module obtains the latest response delay and other information from the response delay record table, dynamically calculates the mean, variance, probability distribution and other statistical information of the response delay, and displays it in real time.

[0112] The flow record storage module is used to persistently store the complete record information of all controller load tests. Specifically, each time a round of load testing is completed, the flow record storage module will store all the record information temporarily stored in the OpenFlow protocol data reporting / distribution record table persistently to the disk, and the flow record processing, statistical analysis and display module can read the record information of historical tests and re-process, analyze and display them, avoiding repeated tests to save test time and improve test efficiency. On the other hand, the flow record processing, statistical analysis and display module also supports the simultaneous retrieval of record information of multiple tests, and the calculation of statistical information such as the mean, variance, and probability distribution of the response delay, so as to make the result analysis more comprehensive and accurate.

[0113] In one embodiment, Figure 6 As shown, a method for automated performance testing of a software-defined network controller is provided, comprising:

[0114] Step 602, by constructing a simulated network including a software switch and two or more virtual hosts, after establishing a connection with the network controller to be tested, a specific type of protocol message is sent at high speed through the virtual host, and the sending rate of the message reported by the software switch is obtained by monitoring and statistics;

[0115] Step 604, according to the test load of the specified network controller to be tested, the port and address of the network controller, and the rate at which the software switch reports messages, the number of software switches is calculated, and according to the number of software switches, a data plane network consisting of software switches and virtual hosts capable of starting the number of software switches is constructed based on a network simulation tool;

[0116] Step 606: Calculate the message reporting rate of each software switch according to the test load and the number of software switches corresponding to the data plane network, and send a specific type of protocol message of the message reporting rate from each virtual host connected to the software switch to trigger the software switch to report the corresponding southbound message protocol according to the message reporting rate;

[0117] Step 608: Perform statistical analysis of southbound protocol message information by monitoring the virtual interface connecting the virtual host and the software switch.

[0118] For step 602, specifically:

[0119] Sl-1: The automatic measurement unit of the soft switch traffic generation capability uses the Mininet network simulation tool to build a simple network consisting of a software switch and two or more virtual hosts.

[0120] S1-2: After waiting for the network to be connected to the controller to be tested, start Tcpdump and start traffic monitoring and recording of the virtual host at the port where the virtual host is connected to the software switch.

[0121] S1-3: Start network communication between virtual hosts and generate communication messages that can trigger OpenFlow messages.

[0122] S1-4: After waiting for the network to run for a period of time, close Tcpdump, terminate the traffic monitoring and recording of the virtual host, and select a specific type of communication message from the traffic record as a test message. Generally, the test message can be an ARP message, a RARP message, and an ICMP message.

[0123] S1-5: Start Tcpdump, start traffic monitoring and recording of OpenFlow messages at the port where the software switch is connected to the controller to be tested, send test messages from the virtual host at a specific rate v1, and continue the test for a specified time s.

[0124] S1-6: After the test duration s, terminate the traffic monitoring and recording of the southbound protocol messages of the software switch, and count the OpenFlow message sending rate v2 of the software switch within the test duration s.

[0125] S1-7: If v2 is consistent with v1, increase v1 and return to step S1-5, otherwise set v2 to the peak sending rate of the software switch OpenFlow message.

[0126] Step 604 is specifically as follows:

[0127] S2-1: Call the soft switch traffic generation capability automatic measurement unit interface to obtain the peak sending rate v2 of the software switch OpenFlow message. Call the given load traffic trigger unit interface to obtain the controller OpenFlow message total load rate v3 specified by the tester.

[0128] S2-2: Substitute v2 and v3 into the following calculation formula to obtain the required number of software switches n;

[0129]

[0130] Where n is the calculated number of software switches, α is the software switch load factor, and a value of 1 indicates the number of software switches required for calculating the peak sending rate. Assuming v2 = 8000pps, v3 = 100000pps, α = 0.8, substitute into the formula It can be calculated that n=16, that is, the test topology is a linear network topology including 16 software switches, each switch connecting two virtual hosts.

[0131] S2-3: Use network simulation tools to build a data plane network consisting of n software switches.

[0132] For step 606, specifically:

[0133] S3-1: Call the soft switch traffic generation capability automatic measurement unit interface to obtain the peak sending rate v2 of the software switch OpenFlow message. Call the data plane network automatic construction unit interface to obtain the current data plane network topology information.

[0134] S3-2: Start traffic monitoring and recording of the virtual host, and complete the network communication between all virtual hosts according to the data plane network topology information.

[0135] S3-3: After waiting for all virtual hosts to complete network communication, close the traffic monitoring and recording of the virtual hosts, select a specific type of protocol message from the traffic record for each software switch as a test message that triggers the OpenFlow message sending rate of the software switch, thereby forming a set of n test messages.

[0136] Specifically, for each switch si (i=1,2…,n) in the topology, let the virtual hosts h1si and h2si connected to the switch communicate, and use the Tcpdump tool to capture the messages generated during the communication process at the port connecting the virtual host to the software switch as test messages pi, and finally form a test message set P={p1,…,pn}.

[0137] Generally, the test message can be an ARP message, a RARP message, and an ICMP message, wherein the ARP and RARP messages can trigger the controller under test to send Packet_out, and the ICMP message can trigger the controller under test to send Flow_mod. Different types of test messages can be selected according to different test purposes. For example, an ARP message is selected as a test message to test the Packet_out response delay of the controller under test.

[0138] S3-4: Set the OpenFlow message sending rate (test message sending rate) for each software switch (and its connected virtual host) in the data plane network topology according to the OpenFlow message sending rate allocation strategy, so that the controller's OpenFlow message total load rate v3 is the sum of the OpenFlow message sending rates of each software switch, that is, v3 = vs1 + vs2 + ... + vsn (vsi ≤ v2, i = 1, 2, ..., n);

[0139] Preferably, the embodiment of the present invention adopts a message sending rate average distribution strategy, and the formula is as follows:

[0140]

[0141] Where n is the number of software switches in the test network, assuming it is 16; v2 is the maximum sending rate of Packet_in messages of a single software switch, assuming it is 8000pps; v3 is the controller load to be tested given by the user, assuming it is 100000pps. Therefore, the specified sending rate of each software switch is 6250pps, and the test message sending rate of the virtual host connected to it is 6250pps.

[0142] S3-5: Start the test. The virtual hosts connected to different software switches obtain their corresponding test messages from the test message set P. The test messages are sent from the designated ports of the virtual hosts connected to the software switches at the specified sending rate through the Tcpreplay replay tool, thereby triggering each software switch to send OpenFlow messages at the set rate, thereby triggering the given load flow of the controller under test. At the same time, the southbound protocol message information statistical analysis unit interface is called to enable monitoring and recording of OpenFlow traffic;

[0143] S3-6: After the end condition is triggered, the test ends, Tcpreplay stops sending test messages, and calls the OpenFlow message information statistics and analysis unit interface at the same time, closes the monitoring and recording of OpenFlow traffic, and thus completes a load simulation test of the controller.

[0144] Preferably, the normal end conditions of the test may include a specified test duration, a specified number of sent messages, a user-initiated termination signal, etc., which may be used alone or in a flexible combination according to actual conditions.

[0145] For step 608, specifically:

[0146] S4-1: When the southbound protocol message information statistical analysis unit intercepts the OpenFlow report message on each virtual interface connected to the software switch and the network controller, it determines whether the specific field of the message needs to be modified according to the content of the message field. If modification is required, the report message is sent after modification, and the timestamp, protocol version, message type, source / destination address, message length, message key fields and other information of the report message are analyzed at the same time to form a southbound protocol data report record, which is added to the southbound protocol data report record table;

[0147] S4-2: When the southbound protocol message information statistical analysis unit monitors the downlink message on each virtual interface connecting the software switch and the network controller, it analyzes the timestamp, protocol version, message type, source / destination address, message length, message key fields and other information of the downlink message, forms a southbound protocol data downlink record, and adds it to the southbound protocol data downlink record table; searches the southbound protocol data reporting record table, finds the southbound protocol data reporting record corresponding to the reporting message that triggers the downlink message, calculates and records the corresponding response delay and other information, and adds it to the response delay record table.

[0148] Continuing with the example above, the network test topology contains 16 software switches, each switch sends a Packet_in message carrying an Arp message to the controller to be tested at a rate of 6250pps, so that the load of the controller to be tested is 100000pps. Under this load condition, the test is performed for 30 seconds, and all OpenFlow protocol messages are recorded by the traffic monitoring and recording module, and stored in the OpenFlow protocol data reporting / distribution record table. During the test, the traffic record processing, statistical analysis and display module continuously reads the record information from the OpenFlow protocol data reporting / distribution record table, filters out the Packet_in message carrying the Arp message and the Packet_out response message carrying the Arp message, and extracts the Xid field and timestamp field in the message. The Xid field is used to pair the Packet_in and Packet_out messages, and the timestamp field is used to calculate the response delay of the paired message. After the pairing is completed, a triple data of (Xid, Packet_in_Timestamp, Packet_out_Timestamp) is formed and added to the response delay record table.

[0149] In addition, the display steps are as follows:

[0150] S5-1: Call the southbound protocol message information statistics analysis unit interface, obtain the response delay and other information from the response delay record table, and display the response delay change information in real time.

[0151] S5-2: Dynamically calculate and display statistical information such as the mean, variance, and probability distribution of the response delay.

[0152] Continuing with the above example, during the test, the traffic record processing, statistical analysis and display module extracts all triplet data from the response delay record table every 1 second. If the value of the Packet_in_Timestamp field or the Packet_out_Timestamp field is 0, it indicates that the Packet_in message and the Packet_out message corresponding to the Xid are not paired successfully. The number of triplet data in this case is counted to calculate the packet loss rate; if the value of the Packet_in_Timestamp field and the Packet_out_Timestamp field is not 0, it indicates that the Packet_in message and the Packet_out message corresponding to the Xid are paired successfully. The response delay of each triplet data in this case is counted to calculate the average, maximum, minimum response delay and other indicators as well as the mean, variance, probability distribution and other statistical information of the response delay. The calculation results will be displayed in real time on the user interaction interface.

[0153] Fig.10It is a flowchart of the analysis, recording and statistical presentation of southbound protocol message information in an embodiment of the present invention, corresponding to step 608 and the display step.

[0154] further, Fig.11 A detailed flow chart of the automated testing method in an embodiment of the present invention that integrates the aforementioned parts of the process is shown.

[0155] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0156] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0157] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A software-defined network controller performance automated testing system, characterized in that: The system comprises: a soft switch flow generation capability automatic measurement unit, a data plane network automatic construction unit, a given load flow trigger unit and a southbound protocol message information statistical analysis unit; The automatic measurement unit for the traffic generation capability of the soft switch builds a simulated network including a software switch and two or more virtual hosts, establishes a connection with the network controller to be tested, and then sends a specific type of protocol message at a high speed through the virtual host, and obtains the sending rate of the message reported by the software switch through monitoring and statistics; The data plane network automatic construction unit calculates the number of software switches according to the test load of the specified network controller to be tested, the port and address of the network controller, and the rate at which the software switch reports messages, and constructs a data plane network consisting of software switches and virtual hosts capable of starting the number of software switches based on a network simulation tool according to the number of software switches; The given load flow triggering unit calculates the message reporting rate of each software switch according to the test load and the number of software switches corresponding to the data plane network, and sends a specific type of protocol message of the message reporting rate from each virtual host connected to the software switch to trigger the software switch to report the corresponding southbound message protocol according to the message reporting rate; The southbound protocol message information statistical analysis unit performs southbound protocol message information statistical analysis by monitoring the virtual interface connecting the virtual host and the software switch.

2. The system according to claim 1, characterized in that The soft switch traffic generation capability automatic measurement unit uses a network simulation tool to construct a simulation network connected by a software switch and more than two virtual hosts; After waiting for the network to connect to the network controller to be tested, start monitoring and recording the traffic of the virtual host; Start network communication between virtual hosts and generate communication messages that can trigger southbound protocol messages; After the preset running time, the traffic monitoring and recording of the virtual host is turned off, and a specific type of communication message is selected from the traffic record as a test message; Enable traffic monitoring and recording of southbound protocol messages on the software switch, send test messages from the virtual host at a specific rate v1, and continue the test for a specified duration s; Disable the traffic monitoring and recording of southbound protocol packets of the software switch, and calculate the southbound protocol packet sending rate v2 of the software switch within the test duration s; If v2 is consistent with v1, then v1 is increased and v2 and v1 are counted again. Otherwise, v2 is set to the peak sending rate of the southbound protocol packets of the software switch.

3. The system according to claim 1, characterized in that The data plane network automatic construction unit calculates the number of software switches according to the test load of the specified network controller to be tested, the port and address of the network controller, and the rate at which the software switches report messages: Where n is the calculated number of software switches, α is the software switch load factor, and a value of 1 indicates that the required number of software switches is calculated based on the peak sending rate.

4. The system according to claim 1, characterized in that The given load flow triggering unit obtains the peak sending rate v2 of the southbound protocol message of the software switch, calls the data plane network automatic construction unit interface, and obtains the current data plane network topology information: Start traffic monitoring and recording of virtual hosts, and complete network communication between all virtual hosts based on the data plane network topology information; After waiting for all virtual hosts to complete network communication, close the traffic monitoring and recording of the virtual hosts, select a specific type of protocol message from the traffic record for each software switch as a test message that triggers the southbound protocol message sending rate of the software switch, thereby forming a test message set containing n test messages; According to the southbound protocol message sending rate allocation strategy, the southbound protocol message sending rate / message sending rate is set for each software switch / connected virtual host in the data plane network topology, so that the total southbound protocol message load rate v3 of the controller is the sum of the southbound protocol message sending rates of each software switch v3 = v s1 +v s2 +…+v sn (v si ≤v2,i=1,2,…,n); The virtual hosts connected to different software switches obtain their corresponding test messages from the test message set and send test messages at a set rate to trigger each software switch to send southbound protocol messages at a set rate, thereby triggering the given load flow of the controller to be tested, and at the same time calling the southbound protocol message information statistical analysis unit interface to start monitoring and recording of southbound protocol traffic.

5. The system according to claim 1, characterized in that When the southbound protocol message information statistical analysis unit intercepts a report message on the virtual interface connected to each software switch and the network controller, it determines whether the specific field of the message needs to be modified according to the message field information. If so, the report message is sent after modification, and the timestamp, protocol version, message type, source / destination address, message length and message key fields of the report message are analyzed to form a southbound protocol data report record, which is added to the southbound protocol data report record table; When the southbound protocol message information statistical analysis unit monitors the sent message on the virtual interface connecting each software switch and the network controller, it analyzes the timestamp, protocol version, message type, source / destination address, message length and message key fields of the sent message, forms a southbound protocol data sending record, and adds it to the southbound protocol data sending record table; Search the southbound protocol data reporting record table, find the southbound protocol data reporting record corresponding to the reporting message that triggers the sending message, calculate and record the corresponding response delay and other information, and add it to the response delay record table.

6. The system according to claim 5, characterized in that Also includes: Display unit; Used to call the southbound protocol message information statistics analysis unit interface, obtain the response delay from the response delay record table, and display the response delay change information in real time; Dynamically calculate and display the statistical information of the mean, variance, and probability distribution of the response delay.

7. A method for automated performance testing of a software-defined network controller, characterized in that: The method comprises: By building a simulated network including a software switch and more than two virtual hosts, after establishing a connection with the network controller to be tested, a specific type of protocol message is sent at high speed through the virtual host, and the sending rate of the message reported by the software switch is obtained by monitoring and statistics; According to the test load of the specified network controller to be tested, the port and address of the network controller, and the rate at which the software switch reports messages, the number of software switches is calculated, and according to the number of software switches, a data plane network consisting of software switches and virtual hosts capable of starting the number of software switches is constructed based on a network simulation tool; According to the test load and the number of software switches corresponding to the data plane network, the message reporting rate of each software switch is calculated, and a specific type of protocol message of the message reporting rate is sent from each virtual host connected to the software switch to trigger the software switch to report the corresponding southbound message protocol according to the message reporting rate; By monitoring the virtual interface connecting the virtual host and the software switch, statistical analysis of southbound protocol message information is performed.

8. The method according to claim 7, characterized in that The method includes building a simulated network including a software switch and two or more virtual hosts, establishing a connection with the network controller to be tested, sending a specific type of protocol message at high speed through the virtual host, and monitoring and counting the sending rate of the message reported by the software switch, including: Use network simulation tools to build a simulated network consisting of a software switch and two or more virtual hosts; After waiting for the network to connect to the network controller to be tested, start monitoring and recording the traffic of the virtual host; Start network communication between virtual hosts and generate communication messages that can trigger southbound protocol messages; After the preset running time, the traffic monitoring and recording of the virtual host is turned off, and a specific type of communication message is selected from the traffic record as a test message; Enable traffic monitoring and recording of southbound protocol messages on the software switch, send test messages from the virtual host at a specific rate v1, and continue the test for a specified duration s; Disable the traffic monitoring and recording of southbound protocol packets of the software switch, and calculate the southbound protocol packet sending rate v2 of the software switch within the test duration s; If v2 is consistent with v1, then v1 is increased and v2 and v1 are counted again. Otherwise, v2 is set to the peak sending rate of the southbound protocol packets of the software switch.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 7 and 8 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 7 and 8 are implemented.

Images