Software-defined network controller performance automated testing system and method

Abstract

This application relates to an automated performance testing system and method for software-defined network controllers. The system includes: an automatic measurement unit for softswitch traffic generation capability, an automatic data plane network construction unit, a given load traffic triggering unit, and a southbound protocol message information statistical analysis unit. By relying on software-defined network simulation tools and widely used software switches, it facilitates testing of network controllers based on constantly evolving and new southbound protocols, ensuring the advanced nature and universality of the testing system.

Description

Technical Field

[0001] This application relates to the field of computer network technology, and in particular to an automated performance testing system and method for software-defined network controllers. Background Technology

[0002] Software-Defined Networking (SDN), as a network architecture and technology, achieves network programmability and flexibility by separating the control plane from the data plane. In SDN, the data plane forwards data packets according to specific policies, while the control plane uses network controllers to achieve centralized control and logical management of the network. As the core component of SDN, the network controller is responsible for managing and controlling the forwarding behavior of the data plane, and its performance directly affects the network's operational efficiency and stability. Controller failure can directly lead to network malfunction. Due to differences in system architecture, implementation technology, and programming languages, network controllers vary in performance. Especially with the rapid growth of network traffic and increasing network complexity, the performance requirements for controllers are also increasing. From the user's perspective, network controller performance is a crucial factor in selecting and evaluating controllers. From the developer's perspective, network controller performance is an important indicator for research and development. Therefore, network controller performance testing is a very important task.

[0003] In an SDN architecture, the network controller communicates with data plane devices to collect network state information, including topology and device status. Based on established network policies such as routing, it generates flow table entries containing matching rules and processing actions for data packets. These flow tables are then distributed to the SDN switches in the data plane via southbound protocols (such as OpenFlow). The SDN switches match and process incoming data packets based on the received flow table entries. If a data packet matches a flow table entry, it is processed according to the action specified in that entry (e.g., forwarding, dropping). When a switch receives a data packet that does not match a flow table entry, or if the action list in a matching flow table entry specifies forwarding the packet to the controller (e.g., Output = CONTROLLER), it encapsulates the packet as a Packet-In message and sends it to the controller. The controller generates Packet-Out or Flow-Mod messages based on network policies and distributes them to the switches to guide packet processing. Packet-Out messages directly instruct the switch how to process a specific data packet, while Flow-Mod messages modify the switch's flow tables to guide subsequent packet processing.

[0004] Among network controller performance metrics, a crucial core performance indicator is the Packet-Out or Flow-Mod message response latency under a given load (e.g., 100,000 packets per second, or 100,000 pps). This requires focused evaluation and testing.

[0005] Currently, there are three main methods for network controller performance testing: one is the testing method based on physical SDN switches; the second is the event-driven simulation testing method; and the third is the process-oriented testing method based on simulation tools.

[0006] The testing method based on physical SDN switches uses physical SDN switches to construct a data plane network to test the network controller. Because the packet-in message rate triggered by a single physical SDN switch is limited (according to our tests on a certain brand of SDN switch, it can only output a few thousand packet-in messages per second), multiple SDN switches are often needed to meet the specified controller load requirements. This method has advantages such as realism, but it suffers from drawbacks such as high cost and cumbersome testing process.

[0007] Event-driven simulation testing methods primarily test SDN controller performance by constructing southbound protocol messages sent from the data plane network to the network controller. This method mainly uses CBench or its derivatives. After running, CBench simulates an SDN switch establishing a connection with the SDN controller. Upon receiving a request message from the SDN controller, CBench consistently replies with pre-defined response messages. CBench can also proactively construct other types of southbound protocol messages and send them to the SDN controller. The advantages of event-driven simulation testing methods are low resource consumption and ease of use. However, due to the continuous evolution of SDN southbound protocols, event-driven simulation testing methods face shortcomings in areas such as the realism of simulated scenarios, limited support for southbound protocols, flexibility of the testing process, accuracy of test results, and scalability of testing tools.

[0008] The procedural testing method based on simulation tools primarily utilizes widely used open-source SDN network simulation tools (mainly Mininet) for testing. First, the tester uses the Mininet network simulation tool to start a corresponding number of software switches and establish corresponding links between them based on pre-configured topology and network controller information, thus constructing and running a data plane network. Second, a packet capture tool needs to be configured to capture specific packets. Then, the tester needs to use other tools to generate packets that trigger the software switches to send Packet-in messages to the SDN controller, and then send these packets to the software switches. Finally, the packet capture results are analyzed to find the time when the software switches send Packet-in messages and the time when the SDN controller sends Packet-out messages or Flow-Mod packets, and the corresponding latency characteristics are calculated. The advantage of this method is its high realism; the software switches used in the network simulation tool run actual software-defined network protocols, and southbound protocol support is good. However, it has disadvantages such as requiring a high level of software-defined network technical expertise from the testers, and the testing process being complex, cumbersome, and time-consuming.

[0009] Existing testing methods have many shortcomings, making it difficult to test performance indicators such as latency characteristics under given load conditions. Overall, there is a current lack of efficient, accurate, fast, and flexible testing methods and tools for network controller performance. Summary of the Invention

[0010] Therefore, it is necessary to provide an automated testing system and method for the performance of software-defined network controllers to address the aforementioned technical problems.

[0011] An automated performance testing system for software-defined network controllers, the system comprising:

[0012] The softswitch traffic generation capability automatic measurement unit, the data plane network automatic construction unit, the given load traffic triggering unit, and the southbound protocol message information statistical analysis unit are all included.

[0013] The automatic measurement unit for the traffic generation capability of the soft switch constructs a simulated network containing one soft switch and two or more virtual hosts. After establishing a connection with the network controller under test, it sends specific types of protocol messages at high speed through the virtual hosts and monitors and statistically obtains the sending rate of the messages reported by the soft switch.

[0014] The automatic data plane network construction unit calculates the number of software switches based on the test load of the specified network controller under test, the port and address of the network controller, and the rate of the packets reported by the software switches. Based on the number of software switches, it constructs a data plane network composed of software switches and virtual hosts that can start the specified number of software switches using a network simulation tool.

[0015] The given load traffic triggering unit calculates the packet reporting rate of each software switch based on the test load and the number of software switches corresponding to the data plane network, and sends a specific type of protocol packet of the packet reporting rate from the virtual host connected to each software switch to trigger the software switch to report the corresponding southbound packet protocol according to the packet reporting rate.

[0016] The southbound protocol message information statistical analysis unit performs southbound protocol message information statistical analysis by monitoring the virtual interface connecting the virtual host and the software switch.

[0017] In one embodiment, the automatic measurement unit for softswitch traffic generation capability uses a network simulation tool to construct a simulated network consisting of a softswitch and two or more virtual hosts.

[0018] Once the network connection with the network controller under test is complete, begin monitoring and recording the virtual host's traffic.

[0019] Initiate network communication between virtual hosts and generate communication messages that can trigger southbound protocol messages;

[0020] After the preset running time, turn off the traffic monitoring and recording of the virtual host, and select specific types of communication packets from the traffic records as test packets;

[0021] Enable traffic monitoring and recording of southbound protocol messages on the software switch, send test messages from the virtual host at a specific rate v1, and continuously test for a specified duration s;

[0022] Disable traffic monitoring and recording of southbound protocol messages of the software switch, and statistically analyze the southbound protocol message sending rate v2 of the software switch within the test duration s.

[0023] If v2 is the same as v1, then v1 is incremented and v2 and v1 are recalculated; otherwise, v2 is set to the peak transmission rate of the southbound protocol messages of the software switch.

[0024] In one embodiment, the data plane network automatic construction unit calculates the number of software switches based on the test load of the specified network controller under test, the port and address of the network controller, and the rate of the packets reported by the software switches:

[0025]

[0026] Where n is the calculated number of software switches, and α is the software switch load factor, with a value of 1 indicating that the number of software switches required is calculated based on the peak transmission rate.

[0027] In one embodiment, the given load traffic triggering unit obtains the peak transmission rate v2 of the southbound protocol packets of the software switch, calls the data plane network auto-construction unit interface, and obtains the current data plane network topology information:

[0028] Start monitoring and recording the traffic of the virtual hosts, and complete a network communication between all virtual hosts based on the data plane network topology information;

[0029] After all virtual hosts have completed network communication, turn off the traffic monitoring and recording of the virtual hosts. Select a specific type of protocol message from the traffic records for each software switch as a test message to trigger the southbound protocol message sending rate of the software switch, thus forming a test message set containing n test messages.

[0030] According to the southbound protocol message transmission rate allocation strategy, the southbound protocol message transmission rate / transmission rate is set for each software switch / connected virtual host in the data plane network topology, so that the total southbound protocol message load rate v3 of the controller is the sum of the southbound protocol message transmission rates of each software switch, v3 = v s1 +v s2 +…+v sn (v si ≤v2,i=1,2,…,n);

[0031] Virtual hosts connected to different software switches obtain their respective test packets from the test packet set and send the test packets at a set rate to trigger each software switch to send southbound protocol packets at a set rate, thereby triggering the given load traffic of the controller under test. At the same time, the interface of the southbound protocol packet information statistical analysis unit is called to start the monitoring and recording of southbound protocol traffic.

[0032] In one embodiment, when the southbound protocol message information statistical analysis unit intercepts the reported message on each virtual interface connected to the software switch and the network controller, it determines whether specific fields of the message need to be modified based on the message field information. If so, the message is modified and then sent again. At the same time, the unit analyzes the timestamp, protocol version, message type, source / destination address, message length, and key fields of the reported message to form a southbound protocol data reporting record and adds it to the southbound protocol data reporting record table.

[0033] When the Southbound Protocol Message Information Statistical Analysis Unit detects a sent message on each virtual interface connected to the software switch and network controller, it analyzes the timestamp, protocol version, message type, source / destination address, message length, and key fields of the sent message to form a Southbound Protocol Data Sending Record, which is then added to the Southbound Protocol Data Sending Record Table. The Unit also searches the Southbound Protocol Data Reporting Record Table to find the Southbound Protocol Data Reporting Record corresponding to the reporting message that triggered the sent message, calculates and records the corresponding response delay and other information, and adds it to the Response Delay Record Table.

[0034] In one embodiment, it further includes: a display unit;

[0035] This is used to call the Southbound Protocol Message Information Statistical Analysis Unit interface, obtain the response delay from the response delay record table, and display the response delay change information in real time.

[0036] The system dynamically calculates and displays the statistical information of the mean, variance, and probability distribution of the response delay.

[0037] An automated performance testing method for software-defined network controllers, the method comprising:

[0038] By constructing a simulated network containing one software switch and two or more virtual hosts, and establishing a connection with the network controller under test, specific types of protocol messages are sent at high speed through the virtual hosts, and the sending rate of the messages reported by the software switch is monitored and statistically obtained.

[0039] Based on the test load of the specified network controller under test, the port and address of the network controller, and the rate of the reported messages of the software switches, the number of software switches is calculated. Based on the number of software switches, a data plane network consisting of software switches and virtual hosts capable of starting the specified number of software switches is constructed using a network simulation tool.

[0040] Based on the test load and the number of software switches corresponding to the data plane network, the packet reporting rate of each software switch is calculated. A specific type of protocol packet of the packet reporting rate is sent from the virtual host connected to each software switch to trigger the software switch to report the corresponding southbound packet protocol according to the packet reporting rate.

[0041] By monitoring the virtual interface connecting the virtual host and the software switch, statistical analysis of southbound protocol message information is performed.

[0042] In one embodiment, it further includes:

[0043] Construct a simulated network consisting of a software switch and two or more virtual hosts using network simulation tools;

[0044] Once the network connection with the network controller under test is complete, begin monitoring and recording the virtual host's traffic.

[0045] Initiate network communication between virtual hosts and generate communication messages that can trigger southbound protocol messages;

[0046] After the preset running time, turn off the traffic monitoring and recording of the virtual host, and select specific types of communication packets from the traffic records as test packets;

[0047] Enable traffic monitoring and recording of southbound protocol messages on the software switch, send test messages from the virtual host at a specific rate v1, and continuously test for a specified duration s;

[0048] Disable traffic monitoring and recording of southbound protocol messages of the software switch, and statistically analyze the southbound protocol message sending rate v2 of the software switch within the test duration s.

[0049] If v2 is the same as v1, then v1 is incremented and v2 and v1 are recalculated; otherwise, v2 is set to the peak transmission rate of the southbound protocol messages of the software switch.

[0050] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program performing the following steps:

[0051] By constructing a simulated network containing one software switch and two or more virtual hosts, and establishing a connection with the network controller under test, specific types of protocol messages are sent at high speed through the virtual hosts, and the sending rate of the messages reported by the software switch is monitored and statistically obtained.

[0052] Based on the test load of the specified network controller under test, the port and address of the network controller, and the rate of the reported messages of the software switches, the number of software switches is calculated. Based on the number of software switches, a data plane network consisting of software switches and virtual hosts capable of starting the specified number of software switches is constructed using a network simulation tool.

[0053] Based on the test load and the number of software switches corresponding to the data plane network, the packet reporting rate of each software switch is calculated. A specific type of protocol packet of the packet reporting rate is sent from the virtual host connected to each software switch to trigger the software switch to report the corresponding southbound packet protocol according to the packet reporting rate.

[0054] By monitoring the virtual interface connecting the virtual host and the software switch, statistical analysis of southbound protocol message information is performed.

[0055] A computer-readable storage medium having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0056] By constructing a simulated network containing one software switch and two or more virtual hosts, and establishing a connection with the network controller under test, specific types of protocol messages are sent at high speed through the virtual hosts, and the sending rate of the messages reported by the software switch is monitored and statistically obtained.

[0057] Based on the test load of the specified network controller under test, the port and address of the network controller, and the rate of the reported messages of the software switches, the number of software switches is calculated. Based on the number of software switches, a data plane network consisting of software switches and virtual hosts capable of starting the specified number of software switches is constructed using a network simulation tool.

[0058] Based on the test load and the number of software switches corresponding to the data plane network, the packet reporting rate of each software switch is calculated. A specific type of protocol packet of the packet reporting rate is sent from the virtual host connected to each software switch to trigger the software switch to report the corresponding southbound packet protocol according to the packet reporting rate.

[0059] By monitoring the virtual interface connecting the virtual host and the software switch, statistical analysis of southbound protocol message information is performed.

[0060] The aforementioned automated performance testing system and method for software-defined network controllers (SDNs) first includes an automatic measurement unit for softswitch traffic generation capability, an automatic data plane network construction unit, a given load traffic triggering unit, and a southbound protocol message information statistical analysis unit. A simulated network containing one softswitch and two or more virtual hosts is constructed using a network simulation tool to test the message transmission rate reported by the softswitch. Then, based on the test load, the number of softswitches is calculated. Based on the number of softswitches, a data plane network consisting of the softswitches and virtual hosts capable of activating that number is constructed using the network simulation tool. For the test load and the number of softswitches corresponding to the data plane network, the message reporting rate of each softswitch is calculated. A specific type of protocol message corresponding to the message reporting rate is sent from the virtual host connected to each softswitch to trigger the softswitch to report the corresponding southbound protocol message according to the message reporting rate. Finally, southbound protocol message information statistical analysis is performed by monitoring the virtual interfaces connected to the virtual hosts and the softswitch. This invention, by relying on software-defined network simulation tools and widely used softswitches, facilitates testing of network controllers based on constantly evolving and new southbound protocols, ensuring the advanced nature and universality of the testing system. Attached Figure Description

[0061] Figure 1 This is a framework diagram of the automated network controller performance testing system proposed in this invention;

[0062] Figure 2 This is a schematic diagram of the automatic measurement unit for traffic generation capability of the soft switch in an embodiment of the present invention;

[0063] Figure 3 This is a schematic diagram of the automatic data plane network construction unit in an embodiment of the present invention;

[0064] Figure 4 This is a schematic diagram of a given load flow triggering unit in an embodiment of the present invention;

[0065] Figure 5 This is a schematic diagram of the southbound protocol message information statistical analysis unit in an embodiment of the present invention;

[0066] Figure 6 This is a flowchart of the overall automated testing method in this embodiment of the invention;

[0067] Figure 7 This is a flowchart of the automatic measurement of the traffic generation capability of the soft switch in an embodiment of the present invention;

[0068] Figure 8 This is a flowchart illustrating the automatic construction of a data plane network in an embodiment of the present invention;

[0069] Figure 9 This is a flowchart of a load test for the controller under test in an embodiment of the present invention;

[0070] Figure 10 This is a flowchart of the southbound protocol message information analysis, recording, and statistical presentation in an embodiment of the present invention;

[0071] Figure 11 This is a detailed flowchart of the automated testing method after integrating the aforementioned processes in this embodiment of the invention. Detailed Implementation

[0072] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0073] In one embodiment, such as Figure 1 As shown, an automated performance testing system for a software-defined network controller is provided, comprising the following steps: an automatic measurement unit for the traffic generation capability of a softswitch, an automatic data plane network construction unit, a given load traffic triggering unit, and a southbound protocol message information statistical analysis unit.

[0074] The automatic measurement unit for generating traffic from a soft switch constructs a simulated network containing one soft switch and two or more virtual hosts. After establishing a connection with the network controller under test, it sends specific types of protocol messages at high speed through the virtual hosts and monitors and statistically obtains the sending rate of the messages reported by the soft switch.

[0075] The automatic data plane network construction unit calculates the number of software switches based on the test load of the specified network controller under test, the port and address of the network controller, and the rate of the packets reported by the software switches. Based on the number of software switches, it constructs a data plane network consisting of software switches and virtual hosts that can be started using a network simulation tool.

[0076] Given the load traffic triggering unit, it calculates the packet reporting rate of each software switch based on the test load and the number of software switches corresponding to the data plane network. It then sends a specific type of protocol packet of the packet reporting rate from the virtual host connected to each software switch to trigger the software switch to report the corresponding southbound packet protocol according to the packet reporting rate.

[0077] The Southbound Protocol Message Information Statistical Analysis Unit performs Southbound Protocol Message Information Statistical Analysis by monitoring the virtual interface connecting the virtual host and the software switch.

[0078] In the aforementioned automated performance testing system for software-defined network controllers, the system first includes an automatic measurement unit for softswitch traffic generation capability, an automatic data plane network construction unit, a given load traffic triggering unit, and a southbound protocol message information statistical analysis unit. A simulated network containing one softswitch and two or more virtual hosts is constructed using network simulation tools to test the message transmission rate reported by the softswitch. Then, based on the test load, the number of softswitches is calculated. Based on the number of softswitches, a data plane network consisting of the softswitches and virtual hosts capable of activating that number is constructed using the network simulation tools. For the test load and the number of softswitches corresponding to the data plane network, the message reporting rate of each softswitch is calculated. A specific type of protocol message at the message reporting rate is sent from the virtual host connected to each softswitch to trigger the softswitch to report the corresponding southbound protocol message according to the message reporting rate. Finally, southbound protocol message information statistical analysis is performed by monitoring the virtual interfaces connected to the virtual hosts and the softswitch. This invention, by relying on software-defined network simulation tools and widely used softswitches, facilitates testing of network controllers based on constantly evolving and new southbound protocols, ensuring the advanced nature and universality of the testing system.

[0079] In one embodiment, the step of automatically measuring the traffic generation capability of the soft switch includes:

[0080] S1, after the network and the network controller under test are connected, start monitoring and recording the traffic of the virtual host.

[0081] S2 initiates network communication between virtual hosts, generating communication messages that can trigger southbound protocol messages.

[0082] S3, after a preset running time, disables traffic monitoring and recording of the virtual host, and selects specific types of communication packets from the traffic records as test packets.

[0083] S4 enables traffic monitoring and recording of southbound protocol messages on the software switch, sends test messages from the virtual host at a specific rate v1, and continuously tests for a specified duration s.

[0084] S5, disable traffic monitoring and recording of southbound protocol messages of the software switch, and count the southbound protocol message sending rate v2 of the software switch within the test duration s.

[0085] S6. If v2 is the same as v1, then increase v1 and recount v2 and v1. Otherwise, set v2 to the peak sending rate of the southbound protocol message of the software switch.

[0086] Specifically, the automatic measurement unit for traffic generation capability of a soft switch tests the peak rate of southbound protocol message transmission of a single soft switch, generates traffic generation capability characteristic information of a single soft switch based on the analysis results, and provides the information to the automatic data plane network construction unit and the given load traffic triggering unit. Figure 2 The diagram shown is an automatic measurement unit for softswitch traffic generation capability according to an embodiment of the present invention. It mainly consists of a softswitch generation module, a test packet capture module, and a softswitch traffic peak test and analysis module.

[0087] The soft switch generation module uses the Mininet network simulation tool and the Open vSwitch software switch tool to generate a test network topology that consists of only a single software switch (the switch connects virtual host h1 and virtual host h2) connected to the controller under test.

[0088] The test packet capture module is used to capture test packets that can trigger the software switch to send OpenFlow protocol packets such as Packet_in. Specifically, this module uses virtual host h1 and virtual host h2 in the test network topology to communicate, and uses the Tcpdump tool to capture the packets generated during the communication process at the port connecting the virtual host and the software switch as test packets. These packets can trigger the software switch's default flow table entries, thereby triggering the software switch to send Packet_in packets.

[0089] The softswitch traffic peak testing and analysis module is used to test the peak transmission rate of OpenFlow protocol packets such as Packet_in from a single softswitch in the current deployment platform environment. Specifically, this module uses the Tcpreplay replay tool to replay captured test packets at a certain rate v1 on the port connected to the virtual host h1 and the softswitch. Simultaneously, the Tcpdump tool monitors and records all OpenFlow protocol packets on the port connected to the controller under test (DUT), filters out the corresponding Packet_in packets, and calculates their transmission rate v2. If v2 is the same as v1, the test packet transmission rate is increased by a specified step_size, making v1 = v1 + step_size. If v2 is less than v1, v2 is considered the peak transmission rate limit of Packet_in packets for a single softswitch in the current deployment platform environment. This testing process is repeated multiple times, obtaining v2 for each test and calculating the average value. Finally, this module uses the calculated average value as the traffic generation capability characteristic information of the softswitch, providing it as input data to the relevant modules of the other units mentioned later.

[0090] In another embodiment, the step of calculating the required number of software switches includes:

[0091] Based on the specified test load of the network controller under test, the port and address of the network controller, and the rate at which the software switches report messages, the number of software switches is calculated as follows:

[0092]

[0093] Where n is the calculated number of software switches, and α is the software switch load factor, with a value of 1 indicating that the number of software switches required is calculated based on the peak transmission rate.

[0094] further, Figure 3 The diagram shown is a schematic of the automatic data plane network construction unit according to an embodiment of the present invention, which only includes the automatic topology analysis and generation module.

[0095] The software switch generation module first obtains the software switch traffic generation capability characteristic information v2 and the user-given load traffic information v3 of the controller under test from the software switch traffic peak test and analysis module and the controller under test traffic triggering module of the given load traffic triggering unit mentioned later, respectively. Then, it substitutes v2 and v3 into the calculation formula. The required number of software switches is determined. Finally, the Mininet network simulation tool and the Open vSwitch software switch tool are used to generate a test topology containing the required number of software switches. This module then provides the test network topology information to the relevant modules of the given load traffic triggering unit mentioned later, for load testing of the controller under test.

[0096] In one embodiment, the steps triggered by a given load traffic include:

[0097] S1, start monitoring and recording the traffic of the virtual host, and complete a network communication between all virtual hosts based on the data plane network topology information;

[0098] S2, after all virtual hosts have completed network communication, turn off the traffic monitoring and recording of virtual hosts, select a specific type of protocol message for each software switch from the traffic records as a test message to trigger the southbound protocol message sending rate of the software switch, thus forming a test message set containing n test messages.

[0099] S3, according to the southbound protocol message transmission rate allocation strategy, set the southbound protocol message transmission rate / transmission rate for each software switch / connected virtual host in the data plane network topology, so that the total southbound protocol message load rate v3 of the controller is the sum of the southbound protocol message transmission rates of each software switch, v3 = v s1 +v s2 +…+v sn (v si ≤v2,i=1,2,…,n);

[0100] S4, virtual hosts connected to different software switches obtain their respective test packets from the test packet set and send the test packets at a set rate to trigger each software switch to send southbound protocol packets at a set rate, thereby triggering the given load traffic of the controller under test. At the same time, the southbound protocol packet information statistical analysis unit interface is called to start the monitoring and recording of southbound protocol traffic.

[0101] Furthermore, the given load traffic triggering unit first captures test packets for each switch in the data plane network that can trigger it to send southbound protocol packets. Then, based on the traffic generation capability characteristics of the software switch and the custom allocation strategy, it allocates a southbound protocol packet sending rate to each software switch. Finally, it triggers the output of reporting packets at the corresponding rate, so that the sum of their rates is equal to the load traffic rate given by the controller set by the user. Figure 4 The diagram shows a given load traffic triggering unit according to an embodiment of the present invention, which mainly consists of a test message capture module and a controller-under-test traffic triggering module. This unit interacts with the other three units and is responsible for the load simulation of the controller-under-test.

[0102] The packet capture module described here has the same function as the packet capture module in the automatic measurement unit for softswitch traffic generation described earlier; both are used to capture test packets that trigger the softswitch to send OpenFlow protocol packets such as Packet_in. The difference is that the packet capture module described earlier captures test packets for a single softswitch to test the peak packet sending rate of the switch; the packet capture module here captures test packets for all softswitch switches in the test network topology, forming a test packet set to trigger the load traffic of the controller under test. In other words, to ensure the authenticity of the test results, the traffic used for testing should be based on the actual generated test topology.

[0103] The traffic triggering module of the controller under test obtains the test packet set from the test packet capture module of this unit, obtains the test network topology information from the topology automatic analysis and generation module of the data plane network automatic construction unit, and obtains the software switch traffic generation capability characteristic information from the software switch traffic generation capability automatic measurement unit. After completing the preparation work for the traffic triggering of the controller under test, it starts packet replay.

[0104] In one embodiment, the southbound protocol message information analysis and recording step includes:

[0105] S1, when a report message is intercepted on each virtual interface connected to the software switch and network controller, it determines whether specific fields of the message need to be modified based on the message field information. If so, the message is modified and then sent. At the same time, the timestamp, protocol version, message type, source / destination address, message length and key fields of the message are analyzed to form a southbound protocol data report record and add it to the southbound protocol data report record table.

[0106] S2, the Southbound Protocol Message Information Statistical Analysis Unit listens for a sent message on each virtual interface connected to the software switch and network controller. It analyzes the timestamp, protocol version, message type, source / destination address, message length, and key fields of the sent message to form a Southbound Protocol Data Sending Record, which is then added to the Southbound Protocol Data Sending Record Table. It also searches the Southbound Protocol Data Reporting Record Table to find the Southbound Protocol Data Reporting Record corresponding to the reporting message that triggered the sent message, calculates and records the corresponding response delay and other information, and adds it to the Response Delay Record Table.

[0107] further, Figure 5 The diagram shows a southbound protocol message information statistical analysis unit according to an embodiment of the present invention. This unit mainly consists of a traffic monitoring and recording module, a traffic recording processing module, a statistical analysis and display module, and a traffic recording storage module, and is used for monitoring and recording OpenFlow protocol messages, analyzing and displaying results, and storing test data.

[0108] When the traffic monitoring and recording module receives an enable / disable traffic monitoring and recording command from the traffic triggering module of the controller under test, it will enable / disable monitoring and recording of OpenFlow protocol messages at all ports connected to the controller under test on all software switches in the test topology. When a reported message is detected, it determines whether specific fields of the message need to be modified. If modification is required, the message is modified before being sent. Simultaneously, it analyzes the timestamp, protocol version, message type, source / destination address, message length, and key message fields of the reported message to form an OpenFlow protocol data reporting record. When a sent message is detected, it analyzes the timestamp, protocol version, message type, source / destination address, message length, and key message fields of the sent message to form an OpenFlow protocol data sent record. These records are formatted into a specific data structure and temporarily stored in the OpenFlow protocol data reporting / sent record table.

[0109] In one embodiment, the steps for displaying southbound protocol message information statistics include:

[0110] Call the Southbound Protocol Message Information Statistical Analysis Unit interface to obtain the response delay from the response delay record table and display the response delay change information in real time; dynamically calculate and display the statistical information of the mean, variance, and probability distribution of the response delay.

[0111] The traffic recording, processing, statistical analysis, and display module reads record information from the OpenFlow protocol data reporting / downloading record table of the traffic monitoring and recording module at fixed intervals. After matching the reported / downloaded records, it calculates the response latency, processes it, and adds the resulting response latency record to the response latency record table. Subsequently, the module retrieves the latest response latency and other information from the response latency record table, dynamically calculates the mean, variance, probability distribution, and other statistical information of the response latency, and displays it in real time.

[0112] The traffic recording storage module is used to persistently store complete record information from each controller load test. Specifically, after each load test, the traffic recording storage module persistently stores all record information temporarily stored in the OpenFlow protocol data reporting / sending record table to disk. The traffic recording processing, statistical analysis, and display module can then read the historical test record information and reprocess, analyze, and display it, avoiding repeated testing, saving test time, and improving test efficiency. Furthermore, the traffic recording processing, statistical analysis, and display module also supports retrieving record information from multiple tests simultaneously to calculate statistical information such as the mean, variance, and probability distribution of response latency, making the result analysis more comprehensive and accurate.

[0113] In one embodiment, such as Figure 6 As shown, an automated performance testing method for software-defined network controllers is provided, including:

[0114] Step 602: By constructing a simulated network containing one software switch and two or more virtual hosts, and establishing a connection with the network controller under test, specific types of protocol messages are sent at high speed through the virtual hosts, and the sending rate of the messages reported by the software switch is monitored and statistically obtained.

[0115] Step 604: Based on the test load of the specified network controller under test, the port and address of the network controller, and the rate of the reported messages of the software switches, calculate the number of software switches. Based on the number of software switches, construct a data plane network composed of software switches and virtual hosts that can start the specified number of software switches using a network simulation tool.

[0116] Step 606: Calculate the packet reporting rate of each software switch based on the test load and the number of software switches corresponding to the data plane network. Send a specific type of protocol packet of the packet reporting rate from the virtual host connected to each software switch to trigger the software switch to report the corresponding southbound packet protocol according to the packet reporting rate.

[0117] Step 608: Perform southbound protocol message information statistical analysis by monitoring the virtual interface connecting the virtual host and the software switch.

[0118] For step 602, the specific steps are as follows:

[0119] Sl-1: The Automatic Measurement Unit for Softswitch Traffic Generation Capability utilizes the Mininet network simulation tool to construct a simple network consisting of a software switch and two or more virtual hosts connected together.

[0120] S1-2: After the network and the controller under test are connected, start Tcpdump and begin monitoring and recording the traffic of the virtual host at the port where the virtual host is connected to the software switch.

[0121] S1-3: Start network communication between virtual hosts and generate communication messages that can trigger OpenFlow messages.

[0122] S1-4: After the network has been running for a period of time, shut down Tcpdump to terminate the virtual host's traffic monitoring and recording. Select specific types of communication packets from the traffic records as test packets. Generally, test packets can be ARP packets, RARP packets, and ICMP packets.

[0123] S1-5: Start Tcpdump to begin monitoring and recording OpenFlow packet traffic at the port where the software switch and the controller under test are connected. Send test packets from the virtual host at a specific rate v1 and continue testing for a specified duration s.

[0124] S1-6: After a continuous test duration of s, terminate the traffic monitoring and recording of southbound protocol messages of the software switch, and statistically analyze the OpenFlow message sending rate v2 of the software switch within the test duration of s.

[0125] S1-7: If v2 is the same as v1, increase v1 and return to step S1-5; otherwise, set v2 to the peak transmission rate of OpenFlow messages of the software switch.

[0126] Step 604, specifically:

[0127] S2-1: Call the automatic measurement unit interface for traffic generation capability of the software switch to obtain the peak transmission rate v2 of OpenFlow packets from the software switch. Call the given load traffic trigger unit interface to obtain the total load rate v3 of OpenFlow packets from the controller specified by the tester.

[0128] S2-2: Substitute v2 and v3 into the following calculation formula to obtain the required number of software switches n;

[0129]

[0130] Where n is the calculated number of software switches, and α is the software switch load factor, with a value of 1 indicating that the number of software switches required is calculated based on the peak transmission rate. Assuming v2 = 8000pps, v3 = 100000pps, and α = 0.8, substitute these values ​​into the formula... It can be calculated that n=16, that is, the test topology is a linear network topology containing 16 software switches, each switch connecting two virtual hosts.

[0131] S2-3: Construct a data plane network containing n software switches using network simulation tools.

[0132] For step 606, the specific details are as follows:

[0133] S3-1: Call the automatic measurement unit interface of the software switch's traffic generation capability to obtain the peak transmission rate v2 of the software switch's OpenFlow packets. Call the automatic data plane network construction unit interface to obtain the current data plane network topology information.

[0134] S3-2: Start monitoring and recording the traffic of the virtual host, and at the same time complete a network communication between all virtual hosts based on the data plane network topology information.

[0135] S3-3: After all virtual hosts have completed network communication, turn off the traffic monitoring and recording of the virtual hosts. Select a specific type of protocol message from the traffic records for each software switch as a test message to trigger the OpenFlow message sending rate of that software switch, thus forming a set containing n test messages.

[0136] Specifically, for each switch si (i = 1, 2, ..., n) in the topology, virtual hosts h1si and h2si connected to that switch communicate with each other. At the same time, the Tcpdump tool is used to capture the packets generated during the communication process at the port where the virtual host is connected to the software switch as test packets pi. Finally, the test packet set P = {p1, ..., pn} is formed.

[0137] Generally, test packets can be ARP, RARP, and ICMP packets. ARP and RARP packets can trigger the controller under test (DUT) to send Packet_out messages, while ICMP packets can trigger Flow_mod messages. Different types of test packets can be selected depending on the test objective. For example, ARP packets can be selected as the test packet to test the Packet_out response latency of the DUT.

[0138] S3-4: Based on the OpenFlow message sending rate allocation strategy, set the OpenFlow message sending rate (test message sending rate) for each software switch (and its connected virtual host) in the data plane network topology, so that the total OpenFlow message load rate v3 of the controller is the sum of the OpenFlow message sending rates of each software switch, i.e., v3 = vs1 + vs2 + ... + vsn (vsi ≤ v2, i = 1, 2, ..., n);

[0139] Preferably, the embodiments of the present invention employ an average allocation strategy for message transmission rates, as shown in the following formula:

[0140]

[0141] Where n is the number of software switches in the test network, assumed to be 16; v2 is the maximum packet_in message sending rate of a single software switch, assumed to be 8000pps; v3 is the load given by the user to the controller under test, assumed to be 100000pps. Therefore, the specified sending rate of each software switch is 6250pps, and the test message sending rate of the virtual host connected to it is also 6250pps.

[0142] S3-5: Start the test. Virtual hosts connected to different software switches retrieve their respective test packets from the test packet set P. Using the Tcpreplay replay tool, the test packets are sent from the specified ports of the virtual hosts connected to the software switches at a specified sending rate. This triggers each software switch to send OpenFlow packets at a set rate, thereby triggering the given load traffic of the controller under test. At the same time, the southbound protocol packet information statistical analysis unit interface is called to start monitoring and recording of OpenFlow traffic.

[0143] S3-6: After the termination condition is triggered, the test ends. Tcpreplay stops sending test messages and calls the OpenFlow message information statistics and analysis unit interface to disable OpenFlow traffic monitoring and recording. This completes one load simulation test of the controller.

[0144] Preferably, the normal termination conditions of the test may include a specified test duration, a specified number of messages sent, or a user-initiated termination signal. These conditions can be used individually or in combination, depending on the actual situation.

[0145] For step 608, the specific steps are as follows:

[0146] S4-1: When the Southbound Protocol Message Information Statistical Analysis Unit intercepts OpenFlow reporting messages on each virtual interface connected to the software switch and network controller, it determines whether specific fields of the message need to be modified based on the message field content. If modification is required, the reporting message is sent again after modification. At the same time, it analyzes information such as the timestamp, protocol version, message type, source / destination address, message length, and key fields of the reporting message to form a Southbound Protocol Data Reporting Record and adds it to the Southbound Protocol Data Reporting Record Table.

[0147] S4-2: When the Southbound Protocol Message Information Statistical Analysis Unit detects a sent message on each virtual interface connected to the software switch and network controller, it analyzes information such as the timestamp, protocol version, message type, source / destination address, message length, and key message fields of the sent message to form a Southbound Protocol Data Sending Record, which is then added to the Southbound Protocol Data Sending Record Table. The Unit also searches the Southbound Protocol Data Reporting Record Table to find the Southbound Protocol Data Reporting Record corresponding to the reporting message that triggered the sent message, calculates and records the corresponding response delay and other information, and adds it to the Response Delay Record Table.

[0148] Continuing with the example above, the network test topology includes 16 software switches. Each switch sends Packet_in packets carrying ARP messages to the controller under test (DUT) at a rate of 6250pps, resulting in a load of 100,000pps on the DUT. The test is conducted for 30 seconds under this load. The traffic monitoring and recording module records all OpenFlow protocol packets and stores them in the OpenFlow protocol data reporting / distribution record table. During the test, the traffic recording processing, statistical analysis, and display module continuously reads record information from the OpenFlow protocol data reporting / distribution record table, filters out Packet_in packets carrying ARP messages and Packet_out response packets carrying ARP messages, and extracts the Xid and timestamp fields from the packets. The Xid field is used for pairing Packet_in and Packet_out packets, and the timestamp field is used to calculate the response delay after pairing. After pairing, a triplet of (Xid, Packet_in_Timestamp, Packet_out_Timestamp) is formed and added to the response delay record table.

[0149] In addition, the specific steps for the demonstration are as follows:

[0150] S5-1: Call the Southbound Protocol Message Information Statistical Analysis Unit interface to obtain information such as response delay from the response delay record table and display the response delay change information in real time.

[0151] S5-2: Dynamically calculate and display statistical information such as the mean, variance, and probability distribution of response delay.

[0152] Continuing with the example above, during the test, the traffic recording, processing, statistical analysis, and display module extracts all triplet data from the response latency record table every second. If the value of either the Packet_in_Timestamp or Packet_out_Timestamp field is 0, it indicates that the Packet_in and Packet_out packets for the corresponding Xid were not successfully paired. The number of triplet data in this case is counted to calculate the packet loss rate. If the values ​​of either the Packet_in_Timestamp or Packet_out_Timestamp field are not 0, it indicates that the Packet_in and Packet_out packets for the corresponding Xid were successfully paired. The response latency of each triplet data in this case is counted to calculate indicators such as average, maximum, and minimum response latency, as well as statistical information such as the mean, variance, and probability distribution of the response latency. The calculation results are displayed in real time on the user interface.

[0153] Figure 10The diagram shows a flowchart of the southbound protocol message information analysis, recording, and statistical presentation in an embodiment of the present invention, corresponding to step 608 and the display step.

[0154] further, Figure 11 A detailed flowchart of the automated testing method in this embodiment of the invention, which integrates the aforementioned processes, is shown.

[0155] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0156] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0157] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. An automated performance testing system for software-defined network controllers, characterized in that, The system includes: an automatic measurement unit for softswitch traffic generation capability, an automatic data plane network construction unit, a given load traffic triggering unit, and a southbound protocol message information statistical analysis unit. The automatic measurement unit for the traffic generation capability of the soft switch constructs a simulated network containing one soft switch and two or more virtual hosts. After establishing a connection with the network controller under test, it sends specific types of protocol messages at high speed through the virtual hosts and monitors and statistically obtains the sending rate of the messages reported by the soft switch. The automatic data plane network construction unit calculates the number of software switches based on the test load of the specified network controller under test, the port and address of the network controller, and the rate of the packets reported by the software switches. Based on the number of software switches, it constructs a data plane network composed of software switches and virtual hosts that can start the specified number of software switches using a network simulation tool. The given load traffic triggering unit calculates the packet reporting rate of each software switch based on the test load and the number of software switches corresponding to the data plane network, and sends a specific type of protocol packet of the packet reporting rate from the virtual host connected to each software switch to trigger the software switch to report the corresponding southbound packet protocol according to the packet reporting rate. The southbound protocol message information statistical analysis unit performs southbound protocol message information statistical analysis by monitoring the virtual interface connecting the virtual host and the software switch; The automatic measurement unit for traffic generation capability of the soft switch uses network simulation tools to construct a simulated network consisting of a soft switch and two or more virtual hosts. Once the network connection with the network controller under test is complete, begin monitoring and recording the virtual host's traffic. Initiate network communication between virtual hosts and generate communication messages that can trigger southbound protocol messages; After the preset running time, turn off the traffic monitoring and recording of the virtual host, and select specific types of communication packets from the traffic records as test packets; Enable traffic monitoring and recording of southbound protocol messages on the software switch, send test messages from the virtual host at a specific rate v1, and continuously test for a specified duration s; Disable traffic monitoring and recording of southbound protocol messages of the software switch, and statistically analyze the southbound protocol message sending rate v2 of the software switch within the test duration s. If v2 is the same as v1, then v1 is incremented and v2 and v1 are recalculated; otherwise, v2 is set to the peak transmission rate of the southbound protocol messages of the software switch.

2. The system according to claim 1, characterized in that, The automatic data plane network construction unit calculates the number of software switches based on the test load of the specified network controller under test, the port and address of the network controller, and the rate of message reporting by the software switches. Where n is the calculated number of software switches. This is the software switch load factor. A value of 1 indicates the number of software switches required to be calculated based on the peak transmission rate, and v3 indicates the total load rate of southbound protocol messages.

3. The system according to claim 2, characterized in that, The given load traffic triggering unit obtains the peak transmission rate of the southbound protocol messages of the software switch, calls the data plane network auto-construction unit interface, and obtains the current data plane network topology information: Start monitoring and recording the traffic of the virtual hosts, and complete a network communication between all virtual hosts based on the data plane network topology information; After all virtual hosts have completed network communication, turn off the traffic monitoring and recording of the virtual hosts. Select a specific type of protocol message from the traffic records for each software switch as a test message to trigger the southbound protocol message sending rate of the software switch, thus forming a test message set containing n test messages. According to the southbound protocol message transmission rate allocation strategy, a southbound protocol message transmission rate or message transmission rate is set for each software switch or connected virtual host in the data plane network topology, so that the total southbound protocol message load rate v3 of the controller is the sum of the southbound protocol message transmission rates of each software switch, v3 = v s1 + v s2 + … + v sn v si ≤v2, i = 1, 2,…, n; Virtual hosts connected to different software switches obtain their respective test packets from the test packet set and send the test packets at a set rate to trigger each software switch to send southbound protocol packets at a set rate, thereby triggering the given load traffic of the controller under test. At the same time, the interface of the southbound protocol packet information statistical analysis unit is called to start the monitoring and recording of southbound protocol traffic.

4. The system according to claim 1, characterized in that, When the Southbound Protocol Message Information Statistical Analysis Unit intercepts a reported message on each virtual interface connected to the software switch and network controller, it determines whether specific fields of the message need to be modified based on the message field information. If so, it modifies the message and then sends it back. At the same time, it analyzes the timestamp, protocol version, message type, source / destination address, message length, and key fields of the reported message to form a Southbound Protocol Data Reporting Record, which is then added to the Southbound Protocol Data Reporting Record Table. When the Southbound Protocol Message Information Statistical Analysis Unit listens for a sent message on each virtual interface connected to the software switch and network controller, it analyzes the timestamp, protocol version, message type, source / destination address, message length, and key fields of the sent message to form a Southbound Protocol Data Sending Record and adds it to the Southbound Protocol Data Sending Record Table. Locate the Southbound Protocol data reporting record table, find the Southbound Protocol data reporting record corresponding to the reporting message that triggered the message delivery, calculate and record the corresponding response delay information, and add it to the response delay record table.

5. The system according to claim 4, characterized in that, Also includes: Display unit; This is used to call the Southbound Protocol Message Information Statistical Analysis Unit interface, obtain the response delay from the response delay record table, and display the response delay change information in real time. The system dynamically calculates and displays the statistical information of the mean, variance, and probability distribution of the response delay.

6. A method for automated performance testing of a software-defined network controller, characterized in that, The method includes: By constructing a simulated network containing one software switch and two or more virtual hosts, and establishing a connection with the network controller under test, specific types of protocol messages are sent at high speed through the virtual hosts, and the sending rate of the messages reported by the software switch is monitored and statistically obtained. Based on the test load of the specified network controller under test, the port and address of the network controller, and the rate of the reported messages of the software switches, the number of software switches is calculated. Based on the number of software switches, a data plane network consisting of software switches and virtual hosts capable of starting the specified number of software switches is constructed using a network simulation tool. Based on the test load and the number of software switches corresponding to the data plane network, the packet reporting rate of each software switch is calculated. A specific type of protocol packet of the packet reporting rate is sent from the virtual host connected to each software switch to trigger the software switch to report the corresponding southbound packet protocol according to the packet reporting rate. By monitoring the virtual interface connecting the virtual host and the software switch, statistical analysis of southbound protocol message information is performed. The process involves constructing a simulated network comprising one software switch and two or more virtual hosts, establishing a connection with the network controller under test, and then transmitting specific types of protocol messages at high speed through the virtual hosts. The transmission rate of the messages reported by the software switch is monitored and statistically analyzed. This includes: Construct a simulated network consisting of a software switch and two or more virtual hosts using network simulation tools; Once the network connection with the network controller under test is complete, begin monitoring and recording the virtual host's traffic. Initiate network communication between virtual hosts and generate communication messages that can trigger southbound protocol messages; After the preset running time, turn off the traffic monitoring and recording of the virtual host, and select specific types of communication packets from the traffic records as test packets; Enable traffic monitoring and recording of southbound protocol messages on the software switch, send test messages from the virtual host at a specific rate v1, and continuously test for a specified duration s; Disable traffic monitoring and recording of southbound protocol messages of the software switch, and statistically analyze the southbound protocol message sending rate v2 of the software switch within the test duration s. If v2 is the same as v1, then v1 is incremented and v2 and v1 are recalculated; otherwise, v2 is set to the peak transmission rate of the southbound protocol messages of the software switch.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method of claim 6.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method described in claim 6.

Images