Method, system, and computer-readable medium for testing a network node using source code

Abstract

According to one method, the method occurs at a network device test apparatus. The method includes: receiving one or more source code files for programming a packet forwarding plane of a network node; analyzing the one or more source code files to determine test metadata, wherein analyzing the one or more source code files to determine the test metadata includes identifying a source code portion indicating a condition for rejecting or accepting a packet and determining the test metadata based on the condition; generating one or more test plans for testing the network node using the test metadata, wherein the one or more test plans define test packets to be sent to the network node for testing the implementation of the condition for rejecting or accepting a packet by the network node; and testing the network node using the one or more test plans.

Description

Technical Field

[0001] The subject matter described herein relates to testing apparatus. More particularly, the present application relates to methods, systems, and computer-readable media for testing network nodes using source code. Background Art

[0002] Network operators generally prefer scalable and cost-effective network architectures. Network function virtualization (NFV) is a network architecture concept that utilizes virtualized network functions (VNFs) (e.g., virtual machines or virtual containers that execute software and programs configured to perform one or more network functions) that can be executed on cloud-based or shared physical resources. By using VNFs to perform network functions, network operators can scale their networks based on current workloads or needs, such as by starting and stopping instances of VNFs as needed. In addition, by using VNFs, the need for customized hardware devices for each network function is reduced.

[0003] Various tools and applications are available for configuring or programming network nodes (e.g., physical network nodes and / or VNFs). A programming language called P4 is one such tool that can be used to program the data plane (e.g., packet processing) of a network node (e.g., a VNF or a programmable network device). The P4 language is designed to be implementation or target independent, protocol independent, and field reconfigurable. For example, the P4 source code can be compiled for many different types of targets, does not have native support for network protocols, and allows operators to change the way a target (e.g., a central programming unit (CPU), a network processor, a field programmable gate array (FPGA), or an application-specific integrated circuit (ASIC)) processes packets after deployment. However, problems may arise when using the P4 language and similar tools. Specifically, network operators need to be able to effectively test network nodes, including any P4 source code-related functions. While traditional test tools can be designed for hardware devices with fixed functions, such tools are less suitable for testing network nodes with reconfigurable features (e.g., through the P4 programming language).

[0004] Therefore, in view of these difficulties, a need exists for improved methods, systems, and computer-readable media for testing network nodes using source code. Summary of the invention

[0005] The present application discloses a method, system and computer-readable medium for testing a network node using source code. According to one method, the method occurs at a network device test device. The method includes: receiving one or more source code files for programming a packet forwarding plane of a network node; analyzing the one or more source code files to determine test metadata, wherein analyzing the one or more source code files to determine the test metadata includes identifying a source code portion indicating a condition for rejecting or accepting a packet and determining the test metadata based on the condition; using the test metadata to generate one or more test plans for testing the network node, wherein the one or more test plans define test packets to be sent to the network node for testing the implementation of the condition for rejecting or accepting a packet of the network node; and using the one or more test plans to test the network node.

[0006] This application relates to the following implementation schemes:

[0007] 1) A method for testing a network node using source code, the method comprising:

[0008] At the network equipment test fixture:

[0009] receiving one or more source code files for programming a packet forwarding plane of a network node;

[0010] analyzing the one or more source code files to determine test metadata, wherein analyzing the one or more source code files to determine the test metadata includes identifying a source code portion that indicates a condition to reject or accept a packet and determining the test metadata based on the condition;

[0011] generating one or more test plans for testing the network node using the test metadata, wherein the one or more test plans define test packets to be sent to the network node for testing implementation of the condition of rejecting or accepting packets by the network node; and

[0012] The network node is tested using the one or more test plans.

[0013] 2) The method of item 1, wherein the one or more source code files include P4 programming language source code.

[0014] 3) The method of item 1, wherein the network node comprises a router, a switch, a network interface card, a programmable network device, or a virtualized network function (VNF).

[0015] 4) The method of item 1, wherein the one or more source code files include protocol structure information, protocol field information or protocol parsing information.

[0016] 5) The method of item 1, wherein the source code portion includes a specified code portion, a verification code portion, a definition code portion, a table key and entry code portion, or a protocol field code portion.

[0017] 6) The method of item 1, wherein analyzing the one or more source code files to determine the test metadata includes using the one or more source code files to generate a parsing data structure indicating a processing state of a protocol parser; traversing a path of the parsing data structure to identify a packet header sequence indicating a valid packet; and using the packet header sequence to generate a test plan defining valid test traffic for testing the network node.

[0018] 7) The method of item 6, comprising:

[0019] A second test plan defining invalid test traffic for testing the network node is generated using the sequence of data packet headers.

[0020] 8) The method of item 6, comprising:

[0021] traversing a second path of the parsed data structure to identify a second sequence of packet headers indicative of a valid packet; and

[0022] A second test plan defining valid test traffic for testing the network node is generated using the second sequence of data packet headers.

[0023] 9) The method of item 1, wherein the network device testing apparatus comprises a source code compiler, wherein the source code compiler converts source code in the one or more source code files into logic for implementing the packet forwarding plane of the network node, wherein the source code compiler sends the logic to the network node for execution.

[0024] 10) A system for testing a network node using source code, the system comprising:

[0025] at least one processor;

[0026] at least one memory; and

[0027] A network device testing device implemented using the at least one processor and the at least one memory, wherein the network device testing device is configured to:

[0028] receiving one or more source code files for programming a packet forwarding plane of a network node;

[0029] analyzing the one or more source code files to determine test metadata, wherein analyzing the one or more source code files to determine the test metadata includes identifying a source code portion that indicates a condition to reject or accept a packet and determining the test metadata based on the condition;

[0030] generating one or more test plans for testing the network node using the test metadata, wherein the one or more test plans define test packets to be sent to the network node for testing implementation of the condition of rejecting or accepting packets by the network node; and

[0031] The network node is tested using the one or more test plans.

[0032] 11) The system of item 10, wherein the one or more source code files include P4 programming language source code.

[0033] 12) The system of item 10, wherein the network node comprises a router, a switch, a network interface card, a programmable network device, or a virtualized network function (VNF).

[0034] 13) The system of item 10, wherein the one or more source code files include protocol structure information, protocol field information or protocol parsing information.

[0035] 14) The system of item 10, wherein the source code portion includes a specified code portion, a verification code portion, a definition code portion, a table key and entry code portion, or a protocol field code portion.

[0036] 15) The system of item 10, wherein the network device testing apparatus is configured to generate a parsing data structure indicating a processing state of a protocol parser using the one or more source code files; traverse a path of the parsing data structure to identify a sequence of packet headers indicating valid packets; and generate a test plan defining valid test traffic for testing the network node using the packet header sequence.

[0037] 16) The system of item 15, wherein the network equipment testing apparatus is configured to use the data packet header sequence to generate a second test plan defining invalid test traffic for testing the network node.

[0038] 17) The system of claim 15, wherein the network device testing apparatus is configured to:

[0039] traversing a second path of the parsed data structure to identify a second sequence of packet headers indicative of a valid packet; and

[0040] A second test plan defining valid test traffic for testing the network node is generated using the second sequence of data packet headers.

[0041] 18) The system of item 10, wherein the network device testing device includes a source code compiler, wherein the source code compiler converts the source code in the one or more source code files into logic for implementing the packet forwarding plane of the network node, wherein the source code compiler sends the logic to the network node for execution.

[0042] 19) A non-transitory computer readable medium comprising computer executable instructions embodied in the computer readable medium, which when executed by at least one processor of a computer, performs the following steps:

[0043] receiving one or more source code files for programming a packet forwarding plane of a network node;

[0044] analyzing the one or more source code files to determine test metadata, wherein analyzing the one or more source code files to determine the test metadata includes identifying a source code portion that indicates a condition to reject or accept a packet and determining the test metadata based on the condition;

[0045] generating one or more test plans for testing the network node using the test metadata, wherein the one or more test plans define test packets to be sent to the network node for testing implementation of the condition of rejecting or accepting packets by the network node; and

[0046] The network node is tested using the one or more test plans.

[0047] According to a system, the system includes a network device testing device. The network device testing device includes at least one processor and a memory. The network device testing device is configured to: receive one or more source code files for programming a packet forwarding plane of a network node; analyze the one or more source code files to determine test metadata, wherein analyzing the one or more source code files to determine the test metadata includes identifying a source code portion indicating a condition for rejecting or accepting a packet and determining the test metadata based on the condition; generate one or more test plans for testing the network node using the test metadata, wherein the one or more test plans define test packets to be sent to the network node for testing the implementation of the condition for rejecting or accepting a packet of the network node; and test the network node using the one or more test plans.

[0048] The subject matter described herein can be implemented in software in combination with hardware and / or firmware. For example, the subject matter described herein can be implemented by software executed by a processor. In an example implementation, the subject matter described herein can be implemented using a computer-readable medium having computer-executable instructions stored thereon, wherein the computer execution steps are controlled when the computer-executable instructions are executed by the processor of the computer. Exemplary computer-readable media suitable for implementing the subject matter described herein include non-temporary devices such as disk memory devices, chip memory devices, programmable logic devices, and application-specific integrated circuits. In addition, the computer-readable medium implementing the subject matter described herein can be located on a single device or computing platform, or can be distributed in multiple devices or computing platforms.

[0049] As used herein, the term "node" refers to a physical computer platform that includes one or more processors and memory.

[0050] As used herein, each of the terms "functionality" and "module" refers to hardware, firmware, or software in combination with hardware and / or firmware for implementing the features described herein. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The subject matter described herein will now be described with reference to the accompanying drawings, in which:

[0052] Figure 1 is a diagram illustrating an example test system for testing network nodes using source code;

[0053] Figure 2 is a diagram illustrating an example code engine (CE) for processing source code;

[0054] Figure 3 is a diagram illustrating an example user interface for importing test metadata exported from source code;

[0055] Figure 4 is a diagram illustrating an example user interface for displaying protocol components defined in source code;

[0056] Figure 5 is a diagram illustrating an example user interface for configuring field values ​​of a test data packet;

[0057] Figure 6 is a diagram illustrating an example user interface for line rate tracking during testing;

[0058] Figure 7 is a diagram illustrating an example user interface for displaying test related statistics; and

[0059] Figure 8 is a diagram illustrating an example process for testing a network node using source code. DETAILED DESCRIPTION

[0060] The subject matter described herein relates to methods, systems, and computer-readable media for testing network nodes using source code.When testing one or more network nodes, it may be desirable to model the tests or related workloads so that the tests are relevant to realistic scenarios and conditions.

[0061] According to some aspects of the subject matter described herein, a network device test apparatus or a related entity may generate test metadata (e.g., test plans, test variables and related test values, test conditions, test actions, etc.) for testing a network device (e.g., a network node). For example, a network device test apparatus may test a virtualized network function (VNF), such as a virtual switch, that is programmed or configured using one or more P4 source code files. In this example, a network device test apparatus or a related entity may receive and analyze a P4 source code file to determine test metadata, such as by analyzing the source code to identify a required header sequence, identifying field values ​​to be tested, and / or identifying positive (e.g., valid) and negative (e.g., invalid) test conditions. Continuing with this example, a network device test apparatus or a related entity (e.g., a data packet generator) may, for example, generate test traffic for testing a network node based on the identified test metadata and / or related test plans (e.g., protocol templates, data packet templates, flow templates, test templates, etc.).

[0062] By using source code to generate test metadata and / or related test plans, a network device test apparatus according to aspects described herein can facilitate the rapid and efficient establishment of one or more tests for a network node. For example, by using the P4 source code of a VNF to generate test metadata, a network device test apparatus according to aspects described herein can quickly identify various aspects of the VNF to be tested (e.g., protocol structure, valid and invalid parameter values, expected processing actions, etc.) and at least some values ​​for testing (e.g., protocol header field values), which can significantly reduce the amount of time required for a human operator to perform test configuration.

[0063] Reference will now be made in detail to example embodiments of the subject matter described herein, examples of which are illustrated in the accompanying drawings. Wherever possible, the same reference numerals are used throughout the drawings to refer to the same or like parts.

[0064] Figure 1 is a diagram illustrating an example test system 100 for testing network nodes using source code. Figure 1, an example environment may include a test system 100 and one or more devices and / or a system under test (SUT) 106. The test system 100 may represent any suitable one or more entities (e.g., one or more computing platforms, nodes, or devices) associated with testing the SUT 106 (e.g., one or more security devices). For example, the test system 100 may generate and send traffic to the SUT 106 and / or receive traffic from the SUT 106, and may analyze one or more performance aspects associated with the SUT 106.

[0065] The SUT 106 may be any suitable one or more entities (e.g., devices, systems, or platforms) for receiving, processing, forwarding, and / or sending one or more messages (e.g., data packets). In some embodiments, the SUT 106 may include a network node, such as a network switch, a network router, a network interface card, a packet forwarding device, or one or more VNFs. For example, the SUT 106 or its VNFs may be software and / or VMs executed on shared resources (e.g., computing, storage, and network resources in a cloud computing environment). In some embodiments, the SUT 106 or related logic (e.g., rules associated with packet forwarding / processing) may be configured or programmed using one or more programming languages ​​(e.g., the P4 language maintained by the P4 Language Alliance).

[0066] In some embodiments, the test system 100 may include a stand-alone tool, a test device, a network device test device or platform, or software executed on one or more processors. In some embodiments, the test system 100 may be a single device or node, or may be distributed across multiple devices or nodes. In some embodiments, the test system 100 may include one or more modules for performing various functions related to testing. For example, the test system 100 may include a traffic (e.g., data packet) generator for generating test traffic and / or a code engine for analyzing the source code of the SUT 106 to identify or determine test metadata for testing the SUT 106.

[0067] The test system 100 may include a test controller (TC) 102, a code engine (CE) 104, and a data store 108. The TC 102 may be any suitable one or more entities (e.g., software executed on a processor, a field programmable gateway array (FPGA), and / or an application specific integrated circuit (ASIC), or a combination of software, FPGA, and / or ASIC) for performing one or more aspects associated with testing the SUT 106 and / or various aspects thereof. In some embodiments, the TC 102 may be implemented using one or more processors and / or memories. For example, the TC 102 may utilize one or more processors (e.g., executing software stored in a memory) to generate test packets for various message flows (e.g., flows or sessions). In another example, the TC 102 may also utilize one or more processors to perform or initiate various tests and / or analyses involving test packets and / or related responses from the SUT 106. In this example, the TC 102 may send instructions to various modules or entities in the test system 100 for controlling (e.g., pausing, restarting, or stopping) a test session.

[0068] CE 104 can be any suitable one or more entities for executing one or more aspects associated with processing source code or related source code files. In some embodiments, CE 104 can use one or more processors and / or memories to implement various functions. For example, CE 104 can compile or interpret source code as intermediate code or machine code to execute at SUT 106. In another example, CE 104 can analyze or check source code to generate test metadata (such as header field value, field name, field size or test condition) that can be used to test SUT 106. In this example, CE 104 can identify specific source code primitives or related code parts, and use the information therein to generate related test metadata for testing other actions defined in packet parsing, packet processing or source code.

[0069] In some embodiments, TC 102 and / or CE 104 may include one or more communication interfaces (e.g., one or more receive port modules and one or more transmit port modules) for interacting with users, modules, and / or nodes. For example, a port module may include a network interface card (NIC) or other suitable hardware, firmware, and / or software for receiving or transmitting data via a port (e.g., a physical or logical communication endpoint).

[0070] In some embodiments, the TC 102 and / or the CE 104 may include or provide a communication interface for communicating with a test operator 110. In such embodiments, the test operator 110 may be any entity (e.g., an automated system or a device or system controlled or controllable by a human user) for selecting and / or configuring various aspects associated with testing and / or generating test-related metrics. For example, a user interface (e.g., an application programming interface (API) and a graphical user interface (GUI)) may be provided for entering or modifying configuration information, such as tests to be performed, types of metrics or statistics to be generated, and / or other settings.

[0071] In some embodiments, one or more user interfaces at the test system 100 can support automation, such as through one or more programming languages ​​(e.g., python, PHP, etc.), representational state transfer (REST) ​​APIs, command lines, and / or web-based GUIs. For example, a test operator 110 can use a web browser to interact with a web-based GUI at a TC 102 or CE 104 for programming or configuring one or more aspects for testing the SUT 106.

[0072] In some embodiments, TC 102 or CE 104 can communicate with test system 100 and / or other related entities to receive test configuration information (e.g., test metadata) that can be used to establish and / or execute one or more test sessions. For example, the test configuration information can be derived or obtained by analyzing a P4 source code file. In this example, the test configuration information can be used to generate a test plan that can be used to generate and send specific traffic and / or flows to SUT 106.

[0073] In some embodiments, the TC 102, CE 104, and / or other entities in the test system 100 may include functionality for accessing a data store 108 or other memory. The data store 108 may be any suitable entity or entities (e.g., a storage device, a memory, a non-transitory computer-readable medium, or a storage system) for maintaining or storing information related to a test. For example, the data store 108 may store packet capture related information, such as time increment information, timestamp related data, and / or other information. In this example, the packet capture related information may be used to determine, derive, or calculate one or more test related statistics, such as a time variation metric indicating scheduling fidelity.

[0074] In some embodiments, the data store 108 may also contain information that can be used to generate statistics and / or metrics associated with one or more aspects of the SUT 106. For example, the data store 108 may contain metrics associated with one or more performance aspects of the SUT 106 during one or more test scenarios. In this example, the data store 108 may maintain a particular set of calculated metrics for a first test session or message flow, and may maintain another set of calculated metrics for a second test session or different message flow. In some embodiments, the data store 108 and / or memory may be located at the test system 100, at another node, or distributed across multiple platforms or devices.

[0075] In some embodiments, the test system 100 or one or more entities therein (e.g., TC 102 and / or CE 104) may be configured to analyze source code or related files associated with the SUT 106 and to generate test metadata for testing the SUT 106. For example, the CE 104 may parse a P4 source code file (e.g., a "switch.p4" file) and may extract or export packet format definitions (e.g., headers, header fields, and other data) and packet processing logic from the P4 source code file. In this example, the CE 104 may use the extracted or exported data to generate test metadata, which may be used (e.g., via a packet generator) to create test traffic and / or automatically generate one or more test plans for testing various aspects of the SUT 106 (e.g., protocol implementations defined in the P4 source code files).

[0076] In some embodiments, the test system 100 or one or more entities therein (e.g., TC 102 and / or CE 104) can examine source code or related files associated with the SUT 106 to identify corner cases (e.g., cases that occur outside of normal operating conditions) and use these identified corner cases to automatically add changes to one or more test plans to test the corner cases. For example, the CE 104 can identify a "definition" code section that indicates valid values ​​for an Ethernet "EtherType" field. In this example, using information from the "definition" code section, the CE 104 can generate test metadata that defines, triggers, or generates packets with invalid "EtherType" values, and can generate test metadata that defines, triggers, or generates packets with valid "EtherType" values.

[0077] In some embodiments, the test system 100 or one or more entities therein (e.g., TC 102 and / or CE 104) may be configured to automatically identify and integrate a "new protocol" defined in source code or related files for test configuration. For example, the test system 100 (e.g., the IxNetwork test system) may include a configuration GUI that can parse test metadata generated from source code. In this example, the test system 100 or a related entity may identify a new protocol defined in the test metadata, and may use the new protocol to configure test traffic for testing the SUT 106. Continuing with this example, the test system 100 or a related entity may call a suitably configured packet generator to generate test traffic, changes in the test traffic, statistics, and traffic tracking that may be used to test the SUT 106 based on the new protocol and / or other test metadata derived from the source code.

[0078] In some embodiments, the test system 100 or one or more entities therein (e.g., TC 102 and / or CE 104) may be configured to monitor the performance of the SUT 106 and / or display performance metrics to the test operator 110. For example, the test system 100 may include a network interface card or a receive port therein for observing responses to test packets from the SUT 106. In this example, the receive port may be configured to perform or monitor various metrics or statistics (e.g., packet loss, latency, jitter, etc.) based on one or more user or system defined trace header fields. Continuing with this example, statistics, performance metrics, and / or other data associated with testing the SUT 106 may be provided or displayed to the test operator 110 via a GUI or other interface.

[0079] In some embodiments, the test system 100 or another entity (e.g., CE 104) can generate a number of test plans (e.g., protocol templates, data packet templates, flow templates, test templates, etc.) or related information. For example, CE 104 or another entity can generate a test plan based on possible combinations of protocol headers defined in a source code file (e.g., a P4 file). In this example, CE 104 or another entity can traverse the relevant parsing graph to access the processing state, from the "start" state to the "accept" or "reject" state. Continuing with this example, by examining the parsing graph of the source code file, CE 104 or another entity can identify the header sequence of one or more paths for testing the parser. In some embodiments, for a depth-first traversal of all paths in the parsing graph, multiple test plans for testing these paths can be generated.

[0080] In some embodiments, the test system 100 or another entity (e.g., CE 104) can identify protocol field values ​​and / or related value selection methods (e.g., patterns) for testing SUT 106. For example, CE 104 or another entity can identify and / or check "verify" and "select" conditions in each parsing state defined in the source code file, and can use these conditions to generate test metadata indicating the values ​​of specific fields in the test data packet used to test SUT 106. In another example, CE 104 or another entity can identify and / or check "keys" and "entries" from a table defined in the source code file, and can use this information to generate test metadata indicating the values ​​of specific fields in the test data packet used to test SUT 106. In another example, CE 104 or another entity can also identify and / or check conditions associated with table actions defined in the source code file, and can use this information to generate test metadata for testing SUT 106.

[0081] In some embodiments, the test system 100 or another entity (e.g., CE 104) can generate one or more test plans for testing the SUT 106 with valid values ​​(e.g., parameter field values), and can also generate one or more test plans for testing the SUT 106 with invalid values. For example, the CE 104 or another entity can generate one or more "reject" test plans that define, trigger, or generate packets that are rejected by the SUT 106. In this example, assuming that the SUT 106 supports a multi-protocol label switching (MPLS) header stack with a depth of three, the CE 104 or another entity can generate a first "reject" test plan that defines, triggers, or generates packets with zero MPLS headers and a second "reject" test plan that defines, triggers, or generates packets with four MPLS headers. Continuing with this example, the SUT 106 should reject the test packets associated with the first and second "reject" test plans. In another example, the CE 104 or another entity can generate one or more "accept" test plans that define, trigger, or generate packets accepted by the SUT 106. In this example, assuming that SUT 106 supports a multi-protocol label switching (MPLS) header stack that is three deep, CE 104 or another entity may generate an "accept" test plan that defines, triggers, or generates packets with three MPLS headers.

[0082] In some embodiments, the test system 100 or another entity (e.g., CE 104) can generate or include source file metadata and / or other information in the test plan. For example, when generating a test plan based on a P4 source code file, CE 104 or another entity can include source file information, source file date, version number, etc. in the test plan.

[0083] In some embodiments, the test plan may also include a description of the plan and an outline of how the plan relates to the test case (e.g., the condition or scenario being tested). For example, the test plan may include useful information to distinguish plans with invalid header field values ​​from test plans with valid header field values. In another example, the test plan may include topology information discovered or derived from the P4 source code file and various data emitted by the CE 104 or an associated compiler, such as the source code file name, import date, version number of the test tool, etc.

[0084] It should be understood that Figure 1 is for illustrative purposes only and may be altered, added to, or removed from the Figure 1 The various nodes described, their locations and / or their functions. For example, some nodes and / or functions may be combined into a single entity.

[0085] Figure 2 is a diagram illustrating an example environment 200 for processing source code to program SUT 106. Figure 2 , the test system 100 may include a CE 104. In some embodiments, the CE 104 may include functionality for processing source code and / or related files, such as a router / parser source code 202 and a switch / parser source code 204. Each of the router / parser source code 202 and the switch / parser source code 204 may represent source code for programming one or more aspects of a network node, such as a router, a switch, a network interface card, a programmable network device, or a VNF. For example, the router / parser source code 202 may be one or more P4 source code files for programming a router and / or its associated data plane (e.g., how it processes and routes packets). In another example, the switch / parser source code 204 may be one or more P4 source code files for programming a switch and / or its associated data plane (e.g., how it processes and switches packets).

[0086] In some embodiments, CE 104 may include functionality for verifying source code and / or related file formats (e.g., P4 source code files). For example, CE 104 may verify that switch / parser source code 204 is from an authorized source and / or may confirm that switch / parser source code 204 can be interpreted or compiled into valid machine code for execution. In this example, if an error is found in the source code or other problems are detected, CE 104 may notify the network operator and / or may store various details.

[0087] In some embodiments, CE 104 may include functionality for compiling and / or interpreting source code into machine code, byte code, or other code (e.g., intermediate code) for implementation or execution at target platform 206. For example, CE 104 may compile or interpret P4 source code into a JavaScript Object Notation (JSON) formatted behavioral model that may be used by a C++ implemented software switch (e.g., bmv2).

[0088] The target platform 206 may represent various entities for executing source code or related logic (e.g., compiled machine code). For example, the target platform 206 may include a general-purpose network programming unit (NPU) (e.g., a network processor), a computing programming unit (CPU), a field programmable gate array (FPGA), a software switch (e.g., executed on one or more general-purpose processors), a system on a chip (SoC), or an application-specific integrated circuit (ASIC). In some embodiments, the target platform 206 may require a compiler (e.g., software embedded in the target platform 206 or CE 104) to map or convert the P4 code into bytes or machine code representing the target machine model. In this example, the bytes or machine code generated will depend on the target platform 206.

[0089] In some embodiments, CE 104 may include functionality for analyzing source code and / or related files (e.g., router / parser source code 202 and switch / parser source code 204) to identify code portions that can be used to derive or obtain test metadata. For example, CE 104 may include a test metadata generator that obtains and / or derives test metadata based on certain keywords or code segments. In this example, the test metadata can be used to generate a test plan and / or related test data packets for testing SUT 106.

[0090] It should be understood that Figure 2 is for illustrative purposes only and may be altered, added to, or removed from the Figure 2 The various nodes, their locations and / or their functions described herein may be combined into a single entity. Figure 2 The router / resolver source code 202 and the switch / resolver source code 204 are disclosed as source code examples, but it should be understood that other source code files may be used to program a programmable network device or VNF, such as a load balancing function, an intrusion detection function, an intrusion protection function, an anti-virus function, an anti-spam function, a firewall function, a switching function, or a routing function.

[0091] Figure 3is a diagram illustrating an example user interface 304 for importing test metadata 302 exported from source code 300. In some embodiments (e.g., where test system 100, TC 102, CE 104, and / or another entity includes or comprises a source code compiler), source code 300 may be directly imported into test system 100 and test metadata 302 may be generated (or exported) from source code 300 by test system 100 or an entity therein (e.g., CE 104).

[0092] In some embodiments, the test system 100, TC 102, and / or another entity may include a user interface 304 for configuring a test session or related aspects (e.g., a test plan). For example, after the CE 104 analyzes the source code 300 and exports and / or generates the test metadata 302, the test operator 110 may select one or more files containing the test metadata 302 through the user interface 304. In this example, after importing the test metadata 302 into the TC 102 using the user interface 304, the test operator 110 may use a test plan configuration GUI (e.g., a template editor) to modify or change the test plan, which defines the test traffic sent to the SUT 106 during the test session. In another example, the test system 100 and / or CE 104 may receive and analyze the source code 300, generate the test metadata 302, and automatically import and / or use the test metadata 302 to create or modify the test plan or perform other test-related configurations in the test system 100.

[0093] refer to Figure 3 , source code 300 may represent code from a P4 file. For example, source code 300 may include information for defining a data plane of a switch. In this example, source code 300 may define a valid packet structure, a protocol structure, relevant header fields, and / or values ​​of one or more fields associated with one or more protocols that the switch may parse and / or process.

[0094] In some embodiments, the test system 100 or a related entity (e.g., CE 104) can analyze the source code 300 to obtain relevant code sections that can be used to generate the test metadata 302. For example, the test system 100 or a related entity (e.g., CE 104) can be configured to identify and examine field data structures in the "header_type" code section (e.g., "header_type ethernet_t" and "header_type foo_t" and "header_type bar_t" sections) and can use these data structures to determine the field names of one or more protocol headers. In another example, the test system 100 can identify and examine the "verify", "define", and "select" code sections in the source code 300 and can use the data therein to identify test conditions and / or determine the values ​​of the protocol fields of the test metadata 302.

[0095] Test metadata 302 may represent data that may be used to configure or define a test plan for testing SUT 106. For example, test metadata 302 may include various data about two protocols (e.g., Foo and Bar) defined or mentioned in source code 300. In this example, test metadata 302 may include a field name, a byte length or size, and a Boolean value indicating whether the field value is to be validated or tested.

[0096] The user interface 304 may allow for importing test metadata 302 derived from the source code 300. For example, after the CE 104 analyzes the source code 300, generates the test metadata 302 derived from the source code 300, and creates a metadata file containing the test metadata 302, the test operator 110 may enter the file name (e.g., "switch.metadata") of the metadata file in an import dialog box of a template editor GUI of, for example, the test system 100 or a related entity (e.g., TC 102). In this example, once the file name is entered, the test system 100 or a related entity may check the file for available protocol templates in the test metadata 302. In another example, the test system 100, TC 102, or another entity (e.g., CE 104) may automatically generate and utilize the test metadata 302 in the test system 102. In this example, before the test metadata 302 is automatically generated and imported, the test operator 110 may use the user interface 300 or another user interface to select a source code file from which to derive or obtain the test metadata 302.

[0097] In some embodiments, the test system 100 or a related entity (e.g., TC 102) can generate and use protocol templates based on the test metadata 302 and / or use imported protocol templates to configure test plans or related information for traffic generators and / or other test entities. For example, the test system 100 or a related entity can generate test traffic for testing the SUT 106 based on the test metadata 302 that indicates valid and / or invalid packets and / or protocol structures based on the imported protocol templates.

[0098] It should be understood that Figure 3 is for illustrative purposes only and may be altered, added to, or removed from the Figure 3 For example, you can remove or modify Figure 3 Some user interface elements and / or features described in .

[0099] Figure 4 4 is a diagram illustrating an example user interface 400 for displaying protocol components defined in source code. In some embodiments, the test system 100, TC 102, and / or another entity may include a user interface 400 for configuring a test session or related aspects. For example, a test operator 110 may view various protocol components (e.g., frame lengths, frame / packet structures, protocol fields, and related values) defined in the source code or in test metadata derived therefrom through the user interface 400.

[0100] In some embodiments, user interface 400 can provide a graphical view for one or more protocols exported or imported from a P4 file. For example, a P4 file can indicate that a frame or packet can include an Ethernet II header, a Foo protocol header, and a Bar protocol header. In this example, test metadata indicating the packet structure can be uploaded or imported into TC 102 and / or another entity, whereby user interface 400 can display the packet structure (e.g., protocol stack) in a "tree view" user interface element.

[0101] It should be understood that Figure 4 is for illustrative purposes only and may be altered, added to, or removed from the Figure 4 For example, you can remove or modify Figure 4 Some user interface elements and / or features described in .

[0102] Figure 5is a diagram illustrating an example user interface 500 for configuring field values ​​of a test data packet. In some embodiments, the test system 100, TC 102, and / or another entity may include a user interface 500 for configuring a test session or related aspects. For example, the test operator 110 may configure or select various values ​​of the protocol header fields for the test session through the user interface 500. In this example, the values ​​used during the test session may be based on one or more value selection modes (e.g., an incrementing mode), where a user may configure the mode by entering a starting value, a step value or increment value, and a count value (e.g., how many times to increment before the starting value starts over).

[0103] In some embodiments, the user interface 600 may allow the value of one or more fields to be selected for a test session or an associated test plan. For example, a user may select a value for a port number field or a flag field of a Bar protocol (e.g., a Bar protocol imported or exported from a P4 file) for testing the SUT 106. In some embodiments, the user interface 600 may provide a list of radio boxes or other user interface elements for selecting and / or configuring a value selection mode. Example value selection modes may be a single value, an incrementing mode, a decrementing mode, a value list, a repeatable random (bit mask) mode, a repeatable random (range) mode, or a random mode (e.g., a non-repeatable mode).

[0104] It should be understood that Figure 5 is for illustrative purposes only and may be altered, added to, or removed from the Figure 5 For example, you can remove or modify Figure 5 Some user interface elements and / or features described in .

[0105] Figure 6 is a diagram illustrating an example user interface 600 for line rate tracking during testing. In some embodiments, the test system 100, TC 102, and / or another entity may include a user interface 600 for monitoring a test session or related aspects. For example, the test operator 110 may monitor the line rate of egress and / or ingress traffic through the user interface 600. In this example, the line rate may be determined based on one or more protocol fields (e.g., the port number field of the Bar protocol).

[0106] In some embodiments, the user interface 600 may allow one or more fields to be selected for line rate tracking. For example, a user may select a port number field of the Bar protocol (e.g., a field imported or exported from a P4 file), which may be used to track ingress and egress traffic. In some embodiments, the user interface 600 may provide a hexadecimal view of one or more received or transmitted packets. For example, the user interface 600 may highlight a specific portion of a packet being tracked (e.g., a port number field and / or another field). In some embodiments, the user interface 600 may display delay segments (e.g., segments indicating different amounts of delay) of ingress and / or egress traffic.

[0107] It should be understood that Figure 6 is for illustrative purposes only and may be altered, added to, or removed from the Figure 6 For example, you can remove or modify Figure 6 Some user interface elements and / or features described in .

[0108] Figure 7 is a diagram illustrating an example user interface 700 for displaying test related statistics. In some embodiments, the test system 100, TC 102, and / or another entity may include a user interface 700 for displaying statistics of a test session or related information. For example, the test operator 110 may view statistics based on field values ​​of the test traffic through the user interface 700.

[0109] In some embodiments, the user interface 700 may include a user interface element that indicates statistics of the egress test traffic based on the port number of the Bar protocol. For example, the test system 100 or a related entity may test the SUT 106 by generating and sending multiple test packets to the SUT 106 (e.g., at line rate). In this example, the test operator 110 or the test plan may indicate one or more user-defined statistics that may be based on field values ​​associated with the protocol defined in the source code file (e.g., a P4 file). Continuing with this example, assuming that some statistics are multi-layered or based on different field values, the user interface 700 may allow the user to, for example, "drill down" statistics or finer-grained test details based on one or more fields (e.g., fields in a protocol imported or exported from a P4 file).

[0110] It should be understood that Figure 7 is for illustrative purposes only and may be altered, added to, or removed from the Figure 7 For example, you can remove or modify Figure 7 Some user interface elements and / or features described in .

[0111] Figure 8800 is a diagram illustrating an example process for testing a network node using source code. In some embodiments, example process 800 or portions thereof may be performed by or at test system 100 (eg, network device test apparatus), TC 102, CE 104, and / or another node or module.

[0112] In step 802, one or more source code files for programming a packet forwarding plane for a network node may be received. For example, CE 104 may receive a P4 source code file for defining a switch and an associated parser. In some embodiments, the network node may include a router, a switch, a programmable network device, or a VNF. In some embodiments, the one or more source code files may include P4 programming language source code. In some embodiments, the one or more source code files may include protocol structure information, protocol field information, or protocol parsing information.

[0113] At step 804, one or more source code files may be analyzed to determine test metadata, wherein analyzing the one or more source code files to determine test metadata may include identifying source code portions that indicate conditions for rejecting or accepting a data packet and determining the test metadata based on the conditions. For example, the test system 100 or a related entity (e.g., CE 104) may be configured to identify "validation" and "selection" conditions in the source code, and these conditions may be used to populate values ​​for specific fields of the test data packet.

[0114] In some embodiments, the source code portion may include a specification code portion, a verification code portion, a definition code portion, a table key and entry code portion, or a protocol field code portion.

[0115] In step 806, the test metadata may be used to generate one or more test plans for testing the network node, wherein the one or more test plans define test packets to be sent to the network node for testing the implementation of the conditions for rejecting or accepting packets by the network node. For example, the test system 100 or a related entity (e.g., a packet generator) may use a test plan to indicate test traffic to be sent to the SUT 106. In this example, the test plan may indicate or define various aspects of the test session, including, for example, a number of media streams, how to determine packet header values ​​(e.g., based on a specified mode or selection method), and which fields and / or statistics to track or record during the test session.

[0116] At step 808, the network node may be tested using one or more test plans.

[0117] In some embodiments, analyzing one or more source code files to determine test metadata may include generating a parsing data structure (e.g., a parsing graph or map) indicating a processing state of a protocol parser using the one or more source code files; traversing a path of the parsing data structure to identify a sequence of packet headers indicating valid data packets; and generating a test plan defining valid test traffic for testing a network node using the sequence of packet headers.

[0118] In some embodiments, after using a data packet header sequence to generate a test plan that defines valid test traffic for testing a network node, a second test plan that defines invalid test traffic for testing the network node can be generated using a data packet header sequence.

[0119] In some embodiments, after traversing a first path of the parsed data structure, a second path of the parsed data structure can be traversed to identify a second packet header sequence indicating a valid packet; and the second packet header sequence can be used to generate a second test plan that defines valid test traffic for testing the network node.

[0120] In some embodiments, the network device testing apparatus may include a source code compiler, wherein the source code compiler converts source code in the one or more source code files into logic for implementing the packet forwarding plane of the network node, wherein the source code compiler sends the logic to the network node for execution.

[0121] It should be understood that various details of the subject matter described herein may be changed without departing from the scope of the subject matter described herein. Furthermore, the foregoing description is for illustrative purposes only and not for limiting purposes, as the subject matter described herein is defined by the claims set forth below.

Claims

1. A method for testing a network node using source code, the method comprising: At the network equipment test fixture: receiving one or more source code files for programming a packet forwarding plane of a network node; identifying a portion of the source code indicating a condition for rejecting or accepting a packet and determining test metadata based on the condition; generating a parsing data structure indicating a processing state of the protocol parser using the one or more source code files; traversing a path of the parsed data structure to identify a sequence of packet headers indicative of a valid packet; generating a test plan defining valid test traffic for testing the network node using the packet header sequence; generating one or more test plans for testing the network node using the test metadata, wherein the one or more test plans define test packets to be sent to the network node for testing implementation of the condition of rejecting or accepting packets by the network node; as well as The network node is tested using the one or more test plans.

2. The method of claim 1, wherein the one or more source code files include P4 programming language source code.

3. The method of claim 1, wherein the network node comprises a router, a switch, a network interface card, a programmable network device, or a virtualized network function (VNF).

4. The method according to claim 1, wherein the one or more source code files include protocol structure information, protocol field information or protocol parsing information.

5. The method of claim 1, wherein the source code portion comprises a specification code portion, a verification code portion, a definition code portion, a table key and entry code portion, or a protocol field code portion.

6. The method according to claim 1, comprising: A second test plan defining invalid test traffic for testing the network node is generated using the sequence of data packet headers.

7. The method according to claim 1, comprising: traversing a second path of the parsed data structure to identify a second sequence of packet headers indicative of a valid packet; as well as A second test plan defining valid test traffic for testing the network node is generated using the second sequence of data packet headers.

8. The method according to claim 1, wherein the network device testing apparatus comprises a source code compiler, wherein the source code compiler converts source code in the one or more source code files into logic for implementing the packet forwarding plane of the network node, wherein the source code compiler sends the logic to the network node for execution.

9. An apparatus comprising a processor, wherein the processor is configured to implement the method of any one of claims 1 to 8.

Images