Data mixing of management data and synthetic traffic data for verification of design under test (DUT)

Abstract

The system (100) may include a verification platform (120) configured to represent a design under test (DUT) (122) to verify operation of the DUT (122). The system (100) may also include a computing system (102) including a data build engine (110) configured to build the management data (210) to configure the DUT (122) in a given manner, and a data mix engine (112). The data mixing engine (112) may be configured to access the synthesized traffic data (220) to test operation of the DUT (122), mix the management data (210) and the synthesized traffic data (220) into a mixed data stream (230), and transmit the mixed data stream (230) comprising the management data (210) and the synthesized traffic data to a verification platform (120) representing the DUT (122) over a physical communication link.

Description

BACKGROUND

[0001] Electronic circuits (such as integrated circuits) are used in almost every aspect of modern society, from automobiles to microwave ovens to personal computers, and so on. Circuit design can involve many steps, referred to as a “design flow.” The specific steps of the design flow often depend on the type of circuit being designed, its complexity, the design team, and the circuit manufacturer or foundry that will manufacture the circuit. Electronic design automation (EDA) applications support the design and verification of circuits before they are manufactured. EDA applications can implement various programs, e.g., functions, tools, or features, to analyze, test, or verify circuit designs at various stages of the design flow. BRIEF DESCRIPTION OF DRAWINGS

[0002] The following detailed description and specific examples are described with reference to certain examples.

[0003] Figure 1 An example of a system that supports mixing of management data and synthetic traffic data for verification of designs-under-test (DUTs) is shown.

[0004] Figure 2 An example of generation of a mixed data stream that includes management data for configuring operation of a DUT and traffic data for testing operation of the DUT is shown.

[0005] Figure 3 An example of mixing management data into a data packet stream that includes synthetic traffic data is shown.

[0006] Figure 4 An example of logic that a system can implement to support mixing of management data and synthetic traffic data for verification of DUTs is shown.

[0007] Figure 5 An example of a computing system that supports mixing of management data and synthetic traffic data for verification of DUTs is shown. DETAILED DESCRIPTION

[0008] With advances in modern technology, various verification techniques have been developed for verifying the operation of devices of almost any type and complexity. Verification platforms are particularly prevalent in the verification, testing, and analysis of integrated circuits (ICs). System-on-a-chip (SoC) designs are becoming increasingly complex, and verification platforms can implement a suite of simulation and emulation capabilities to test SoCs as a design under test (DUT). Hardware emulators, software-based simulation systems, field programmable gate array (FPGA) prototyping, and other various techniques can be used to support design verification of increasingly complex IC devices. A verification platform can represent or otherwise contain a DUT for testing and verification through any such techniques. For example, a verification platform can represent (e.g., implement a representation of) a DUT through behavioral models, through FPGA or other hardware-based simulation, through software simulation, etc.

[0009] Modern verification systems are used for verification beyond the scope of traditional SoC and chip designs. As one example, verification platforms can support testing and verification of DUTs in the form of communication networks (and underlying networks), such as cellular network devices (including radios, fronthaul networks and their components, distributed units, antennas, routing structures), and any other type of network device. Network speed, bandwidth, capacity, and communication capabilities continue to rapidly advance, and verification of communication networks and network devices (including network application-specific integrated circuits (ASICs)) can require increasingly powerful verification platforms.

[0010] One challenge in verification of DUTs that are network devices, such as 5G fronthaul network devices, is that configuration of the DUT represented by the verification platform can be a tedious and complex task. For fronthaul network devices and other network device DUTs that are verified via simulation or emulation by a verification platform, options for DUT configuration (e.g., configuration of a 5G radio unit or other network device DUT) are limited. In some cases, a user (e.g., a test engineer) can directly configure a DUT via physical access to the verification platform. However, manual DUT configuration can require direct access to the verification platform itself for even simple DUT parameter changes, which can be cumbersome, error-prone, and inefficient.

[0011] In particular, as an option for DUT configuration in the validation of network equipment, one possibility is to leverage the management plane (M-plane) of network equipment implemented according to Open Radio Access Network (O-RAN) specifications. However, conventional validation platforms provide limited options for communicating through the management plane to configure DUT operation. Users can develop private or proprietary testbenches on their own for DUT validation, and such testbenches can include specially generated M-plane data to configure the DUT for a particular use. However, these testbenches tend to differ from one another, and they are implemented by different third-party solutions, leveraging different software protocols and products from the validation platform.

[0012] Further, the management data of the M-plane can be stateful in nature, as the management data can change or configure the internal state of the network equipment (or downstream equipment). Generating such stateful data for configuring the DUT can vary greatly based on the particular DUT being validated and the particular validation being performed on the DUT. This shortcoming can be particularly limiting when real-time protocol or other stateful validation is required. In some conventional solutions, such as virtual Ethernet, a synthetic generator can be used for packet generation for stateful protocols, e.g., a scripted sequence of packets is configured to generate TCP / IP packets with customizable fields to provide to the DUT for configuration purposes. While a scripted sequence of commands can generate management data through the customizable fields of the transmitted packets, such techniques cannot support validation of DUT communication exchanges in real-time stateful scenarios. Virtual validation implementations are also subject to similar limitations, as scripted sequences of packets cannot validate the order and completeness of all stateful responses from the DUT.

[0013] Another shortcoming of conventional validation systems is that M-plane communication can be limited to a dedicated communication port in the validation system, such as a dedicated Ethernet port of the validation platform, through which M-plane traffic is communicated to directly alter DUT configuration and parameter registers. This design limits the efficiency of the validation platform by requiring a dedicated communication channel, and this limitation becomes increasingly severe when network equipment validation requires a large amount of communication bandwidth to properly test the performance and behavior of the equipment. Such ports can also require additional hardware, such as a speed bridge, to coordinate the timing of the M-plane data. Additionally, many conventional methods for communicating M-plane data to configure the DUT cannot be performed dynamically, e.g., dynamically during the validation process of the network equipment, or dynamically simultaneously with the transmission of network traffic data through the same network channel to test the performance of the network equipment. Similar limitations can exist for DUT configuration of other types of equipment and DUTs in addition to M-plane data for configuring O-RAN network equipment.

[0014] The disclosure herein provides systems, methods, devices, and logic for using data mixing of management data and synthetic traffic data to validate DUTs. The data mixing techniques disclosed herein can support the mixing of multiple data sources for communication over a common communication channel. As a specific example, the data mixing techniques described herein can mix management data used to configure the DUT with traffic data used to test the DUT. By utilizing the communication channel that transmits traffic data to the validation platform and the DUT, mixing management data into such a data stream allows for dynamic updates to the configuration, operation, or state of the DUT (e.g., one or more devices represented by the DUT). Reactive dependencies between data traffic, management data, and DUT functionality can be implemented to more comprehensively test the DUT system and better simulate real-world scenarios and operations of the devices and systems represented by the DUT.

[0015] This document will describe in more detail these and other data blending features and technical advantages of this disclosure.

[0016] Figure 1 An example of a system is shown that supports the mixing of management data and synthetic flow data for the verification of a design under test (DUT). System 100 can provide verification capabilities for the DUT, such as through simulation-based or model-based verification. Figure 1 In the example shown, system 100 includes computing system 102 and verification platform 120. Verification platform 120 may include a combination of hardware and software that supports simulation or emulation of the physical device represented by verification platform 120 in the form of a DUT.

[0017] For example, the verification platform 120 can take the form of a hardware-assisted verification platform, providing various device verification functions, such as virtual platforms, hardware simulation capabilities, and field-programmable gate array (FPGA) prototyping technology. Figure 1 In the example, hardware verification platform 120 represents DUT 122, and verification platform 120 can do so using any supported verification technology to emulate or simulate a system or network device (e.g., system-on-a-chip, network device, communication network, etc.) that DUT 122 may present itself as. Accordingly, DUT 122 may take the form of a specific network device or network system, and verification platform 120 can therefore provide any suitable combination of verification technologies for verifying the system, network, or device as DUT 122.

[0018] exist Figure 1In the example of FIG. 1, system 100 also includes a computing system 102. Computing system 102 can be linked to verification platform 120 through any suitable communication network (e.g., a direct network link or through intermediate network devices and networks). In some implementations, computing system 102 can take the form of a user terminal through which testing, configuration, verification, management, or any other operation on a DUT represented by verification platform 120 can be controlled or initiated. In this regard, computing system 102 can provide any number of interface functionalities through which a user can control testing, verification, and analysis of a DUT represented by verification platform 120. Computing system 102 can take the form of a single or multiple computing devices, such as a computing node, a desktop or laptop computer, a smartphone or other mobile device, a tablet computer, an embedded controller, and the like. In some implementations, computing system 102 can take the form of a workstation supporting suitable operations of a test bench execution, DUT configuration, DUT behavior analysis, or other support of DUT-based verification of a device.

[0019] The data mixing techniques of the present disclosure can be implemented by any computing system that interacts with verification platform 120, such as computing system 102 in FIG. 1. Figure 1 In this regard, computing system 102 can provide a combination of any of the data mixing functionalities described herein to mix management data for configuring characteristics of a DUT (including device components of DUT 122) with traffic data for testing operation of DUT 122. Through the various functionalities described herein, the data mixing techniques of the present disclosure can provide support for dynamic adjustments to DUT configuration, state, and protocols supported by the DUT, support reactive dependencies between DUT configuration and traffic data to test DUT 122, and improve efficiency of communication and configuration of the DUT through a common communication channel with test data.

[0020] As an example implementation supporting any combination of the data mixing functionalities described herein, Figure 1 Computing system 102 is shown to include data construction engine 110 and data mixing engine 112. Engines 110 and 112 (including components thereof) can be implemented in various ways by computing system 102, for example as hardware and programming. Programming of engines 110 and 112 can take the form of processor-executable instructions stored on a non-transitory machine-readable storage medium, and hardware of engines 110 and 112 can include a processor to execute these instructions. The processor can take the form of a single- or multi-processor system, and in some examples computing system 102 implements multiple engines using the same computing system features or hardware components (e.g., a general purpose processor or a general purpose storage medium).

[0021] In operation, the verification platform 120 can represent the DUT 122 to verify operation of the DUT 122 (e.g., verification operations of a DUT in the form of a system or device, such as a network device). In operation, the data construction engine 110 can construct management data to configure the DUT 122 in a given manner. In operation, the data mixing engine 112 can access synthetic traffic data to test operation of the DUT 122, mix the management data and the synthetic traffic data into a mixed data stream, and transmit the mixed data stream including the management data and the synthetic traffic data to the verification platform 120 representing the DUT 122 over a physical communication link.

[0022] These and other features of the data mixing techniques of the present disclosure will be described in greater detail below. Many of the examples described herein are of a DUT in the form of a communication network component, e.g., a fronthaul network device or an underlay network device, such as a radio, antenna, etc. Herein, the term DUT, sometimes referenced in the context of a verification platform, can be used interchangeably with one or more network devices that the DUT can represent. Such network device DUT examples are provided for illustrative purposes, and the data mixing techniques described herein can be implemented and applied consistently to configuration, management, control, and testing of any device type or system that a DUT and verification platform are configured to verify. Some examples of management data for DUT configuration are described herein, but the data mixing techniques of the present disclosure can be applied consistently to mixing any type of live traffic with synthetic traffic data for communication over a common channel.

[0023] Figure 2 An example is shown of generation of a mixed data stream including management data for configuring operation of a DUT and traffic data for testing operation of the DUT. Figure 2 Various example functions are described and implemented by the data construction engine 110 and the data mixing engine 112, but any suitable implementation is contemplated herein. A computing system can implement the functions described in Figure 2 to support any of the data mixing functions described herein.

[0024] To support the data mixing techniques of the present disclosure, the data construction engine 110 can construct management data to configure a DUT, such as a network device, e.g., a radio, antenna, etc. Figure 2The management data built by the data build engine 110 can include any data that configures a characteristic, operation, state, supported protocol, or behavior of the DUT 122. Examples of management data that the data build engine 110 can generate for configuring the DUT 122 include: auto-negotiation parameters, device advertisement functions (e.g., capability notification messages, modes, etc.), time updates, IP address assignments, configuration and advertisement of supported security suites and encryption schemes, software updates for network devices represented by the DUT (e.g., for radios or other devices belonging to the DUT 122). For O-RAN compliant network systems or devices represented by the DUT 122, the data build engine 110 can generate management plane data to configure any network devices belonging to or forming the DUT 122 (e.g., radios, antennas, or other network components).

[0025] The data build engine 110 can generate management data according to any number or combination of different protocols and services. As an illustrative example, the DUT 122 can take the form of a radio unit design to verify fronthaul port communications (e.g., Ethernet) to or from the DUT 122. The data build engine 110 can support the build of management data that maintains an M-plane link with the DUT 122 through different communication services and protocols (e.g., Internet Control Message Protocol (ICMP), Secure Shell (SSH), Dynamic Host Configuration Protocol (DHCP), and any other suitable communication service or protocol). To support multiple protocols through the management data built for the DUT 122, the data build engine 110 can implement, instantiate, or otherwise utilize multiple virtual network interfaces, such as those labeled “VIF1,” “VIF2,” and “VIF3” in FIG. 1. The virtual network interfaces can stimulate the DUT 122 through the generated management data (e.g., protocol data units generated specific to a particular protocol or service provided by a given virtual network interface). Figure 2 The data build engine 110 can generate management data for the DUT 122 through the virtual network interfaces. For example, the data build engine 110 can generate management data for the DUT 122 through the VIF1 interface, the VIF2 interface, and the VIF3 interface. The management data generated by the data build engine 110 can include any data that configures a characteristic, operation, state, supported protocol, or behavior of the DUT 122. Examples of management data that the data build engine 110 can generate for configuring the DUT 122 include: auto-negotiation parameters, device advertisement functions (e.g., capability notification messages, modes, etc.), time updates, IP address assignments, configuration and advertisement of supported security suites and encryption schemes, software updates for network devices represented by the DUT (e.g., for radios or other devices belonging to the DUT 122). For O-RAN compliant network systems or devices represented by the DUT 122, the data build engine 110 can generate management plane data to configure any network devices belonging to or forming the DUT 122 (e.g., radios, antennas, or other network components). n

[0026] ​The data build engine 110 can support the generation of any type of management data according to a particular protocol, service, or configuration to control the operation of the DUT 122. In some implementations, the data build engine 110 can provide a user interface through which a user can select a particular protocol, feature, or configuration to configure the DUT 122. In response to these selections, the data build engine 110 can generate management data to fulfill the user selections. For example, the data build engine 110 can build management data according to user input to configure a network device (e.g., a radio) in a given manner (e.g., according to a particular ICMP configuration, an auto-negotiation scheme, encryption capabilities, or any other suitable configuration of a network device) for validation by the DUT 122.

[0027] In Figure 2 In the example of FIG. 2, the data build engine 110 generates management data 210 that the data mix engine 112 can mix with other traffic data communicated to the DUT 122 to test the DUT 122. Figure 2 Such an example is illustrated in FIG. 2 by synthetic traffic data 220 that the data mix engine 112 can access to mix with the management data 210. As used herein, the synthetic traffic data 220 can include any data intended to test the operation of any one or more devices or one or more systems (such as the DUT 122) represented by the validation platform as presented by the DUT. For a DUT in the form of a fronthaul network device or a wireless network device, the synthetic traffic data 220 can include radio traffic such as user plane traffic that can be communicated over a communication network and processed by a fronthaul network or control plane traffic that controls different radio operations (e.g., scheduling, beamforming, etc.).

[0028] Accordingly, the synthetic traffic data 220 can take the form of data of a data plane, data of a control plane, or a combination of both. The synthetic traffic data 220 can lack (e.g., not include) stateful traffic and thus be unable to change the state of a downstream network device. In some implementations, the synthetic traffic data 220 can be generated locally, e.g., via a script or other data generation technique locally at the computing system 102. Thus, the synthetic traffic data 220 need not be generated by real users of a communication network, but can take the form of script-generated traffic data from an iterative execution of a locally run script or from a synthetic traffic generator (e.g., traffic generation logic) implemented by the computing system 102.

[0029] The data mixing engine 112 can mix the management data 210 and the synthetic traffic data 220 together for communication over a common communication channel. In doing so, the data mixing engine 112 can generate a mixed data stream that includes the management data 210 and the synthetic traffic data 220. In Figure 2 In the illustrated example, the data mixing engine 112 generates a mixed data stream 230 that can include the management data 210 and the synthetic traffic data 220. The data mixing engine 112 can then transmit the mixed data stream 230 to the DUT 122. As used herein, a "communication of the mixed data stream 230" can refer to the actual transmission of the mixed data stream 230 over a physical communication link, or any operation that supports the transmission of the mixed data stream 230 over a physical communication link. Thus, the data mixing engine 112 can communicate the mixed data stream 230 by physically transmitting packets of the mixed data stream 230 over a communication link, passing the mixed data stream 230 to an interface (e.g., a software interface or a network interface) of the computing system 102 for a next step in the communication process, or performing any other suitable operation that supports the transmission of the mixed data stream 230 to the verification system.

[0030] The mixed data stream 230 can support both testing and configuration of the DUT 122 over a single communication channel (e.g., a single data port or communication line). The synthetic traffic data 220 of the mixed data stream 230 can allow for testing of operations of the DUT 122 (e.g., parsing, routing, or otherwise processing communication traffic of the synthetic traffic data 220), and can configure the DUT 122 through the management data 210. Thus, the verification platform 120 can verify the operation of a device by simulating or emulating the operation of a DUT represented by the verification platform 120. The verification platform 120 can support verification through the DUT 122 in accordance with the DUT configuration specified through the management data 210 and the DUT operation testing provided through the synthetic traffic data 220.

[0031] By mixing the management data 210 with the traffic data stream, the data mixing engine 112 can support real-time configuration of the DUT 122, e.g., during a live test session. Rather than cumbersome access through third-party protocols, or directly altering the DUT registers via direct physical access to the verification platform 120, the data mixing engine 112 can provide an efficient, flexible, and elegant mechanism by which configuration of the DUT according to the management data 210 can be performed concurrently or otherwise alongside the DUT testing process. Thus, the data mixing engine 112 can support real-time or reactive testing techniques by mixing the management data 210 to change the configuration of the DUT 122 during testing, e.g., during transmission of the synthetic traffic data 220, and analyzing the behavior and responses of the DUT 122 after changing the configuration. In some examples, the synthetic traffic data 220 can be adjusted to test specific operations of the DUT 122 for specific or real-time changes to the DUT configuration implemented through the mixed management data. The data mixing techniques of the present disclosure can support adaptive and dynamic configuration and testing of the DUT that traditional techniques cannot achieve.

[0032] To support real-time or dynamic DUT configuration changes, the data mixing engine 112 can generate the mixed data stream 230 in a progressive manner. To further explain, the generation and transmission of the mixed data stream 230 can be performed sequentially (e.g., as a sequential data stream, such as a stream of data packets). Doing so can allow the data mixing engine 112 to continuously transmit the synthetic traffic data 220, and interleave the management data 210 into the traffic data stream at specified times. In such examples, the data mixing engine 112 need not generate the entire mixed data stream 230 prior to transmission, but can instead progressively mix and build the mixed data stream 230 as interleaved data including the synthetic traffic data 220 and the management data 210. This means that the data mixing engine 112 can support dynamic generation and mixing of the management data 210 even after transmission of the synthetic traffic data 220 has begun.

[0033] As an illustrative example, the data build engine 110 can build the management data 210 after a first portion of the synthetic traffic data 220 has been transmitted to the DUT 122 of the validation platform 120. This can occur in response to a user selection to test or alter a given configuration of the DUT 122 in real-time or during execution of a test bench. The data mixing engine 112 can mix the built management data 210 with another portion of the synthetic traffic data 220 that is different from the first portion of the synthetic traffic data 220 that has been transmitted to the validation platform 120. Since the synthetic traffic data 220 can be generated locally through a scripting process, the generation and transmission of the synthetic traffic data 220 can occur in parallel with the build of the management data 210, and the data mixing engine 112 can mix the built management data 210 with the accessed synthetic traffic data 220. Thus, the mixed data stream 230 can take the form of a data packet stream that is transmitted over a communication link, including the management data 210 and the synthetic traffic data 220 for the DUT 122.

[0034] In some validation scenarios, the synthetic traffic data 220 includes a much larger amount of data to be transmitted than the management data 210. For example, the mixed data stream can contain 99.99% synthetic traffic data 220 and 00.01% management data 210 for transmission over a common communication channel. Thus, a significant portion of the communication time and resources of the computing system 102 can be dedicated to the transmission of the synthetic traffic data 220 to the validation platform 120. The data mixing engine 110 can mix the management data 210 with the synthetic traffic data 220 so as not to interfere with the timing of the transmission of the synthetic traffic data 220 to the DUT 122, such as by utilizing idle periods of the transmission of the synthetic traffic data 220. An example of this will be described next with reference to Figure 3 Example features of such techniques are described in more detail.

[0035] Figure 3 An example of mixing the management data into a data packet stream that includes synthetic traffic data is shown. In the example of FIG. 3, the data mixing engine 112 can generate the mixed data stream 230 as a data packet stream, as shown as the data packet stream 310. The data packet stream 310 can include data packets of both the management data 210 and the synthetic traffic data 220, and thus, the mixing and communication of the mixed data stream 230 by the data mixing engine 112 can be implemented through the generation and transmission of the data packets (and the operations that support this process). Figure 3 Figure 3

[0036] ​​To further illustrate, synthetic traffic data 220 can include a large amount of data relative to management data 210. Moreover, synthetic traffic data 220 can include timing requirements for communication, e.g., according to Ethernet or other communication protocol timing. Thus, transmission of packet data of synthetic traffic data 220 can require specific line timing through a communication channel, e.g., an Ethernet line. Data mixing engine 112 can support mixing of management data 210 with packet timing of synthetic traffic data 220, and do so without interfering with communication timing requirements of synthetic traffic data 220, e.g., without causing changes in packet timing.

[0037] In doing so, data mixing engine 112 can utilize idle periods for communication of synthetic traffic data 220. For example, data mixing engine 112 can identify idle periods on a physical communication link or any other suitable idle periods within which packet transmission of synthetic traffic data 220 is not scheduled to occur. Data mixing engine 112 can determine idle periods in any suitable manner, e.g., according to a schedule queue of data transmissions. Timing of packet transmissions can be specified via packet data, e.g., header data, which data mixing engine 112 can parse to determine suitable idle times. During idle periods on such a communication link, data mixing engine 112 can insert management data 210 for communication with verification platform 120 to configure DUT 122. For example, data mixing engine 112 can generate a packet including management data 210, and schedule the generated management packet for communication through the physical communication link during an idle period of transmission of synthetic traffic data 220.

[0038] Thus, in the example of Figure 3 In the example of FIG. 3, mixed data stream 230 includes a packet stream 310 that can be transmitted by computing system 102 through a physical communication link to verification platform 120. Data mixing engine 112 can mix management data 210 and synthetic traffic data 220 into mixed data stream 230 by inserting management data 210 into packet stream 310 during idle periods of transmission of synthetic traffic data 220. Communication of such mixed data stream 230 by data mixing engine 112 can take the form of packet scheduling of management packets and synthetic traffic packets, or any other suitable operation that supports transmission of packets forming packet stream 310 of mixed data stream 230.

[0039] The transmission of idle periods of management data packets is just one example of how the data mixing engine 112 can mix management data 210 with synthetic traffic data 220. Any suitable mixing technique is contemplated herein, and the data mixing engine 112 can implement such techniques. As another example, the data mixing engine 112 can apply a priority-based mixing scheme in which data packet priorities are assigned to control the scheduling of data packet transmissions. The data mixing engine 112 can assign or otherwise identify priorities for the management data 210 and the synthetic traffic data 220 (including various priorities even among different data packets or data units of the management data 210 and / or the synthetic traffic data 220). The data mixing engine 110 can then transmit the mixed data stream by scheduling data packet transmissions based on the assigned priorities of the data packets of the management data 210 and the synthetic traffic data 220.

[0040] Many data mixing functions have been described herein by way of various illustrative examples presented by way of various figures, and the data building engine 110 and the data mixing engine 112 can implement a combination of any of the data mixing functions described herein. As another example function, the data mixing engine 112 can support routing communications from the DUT 122, e.g., in response to the transmission of the mixed data stream 230. In these examples, the data mixing engine 112 can provide a routing function, e.g., by routing response data of the synthetic traffic data 220 (e.g., egress data) to a traffic analyzer or other validation component of the computing system 102. For response data provided by the DUT 122 in response to the management data 210 (e.g., stateful response data), the data mixing engine 112 can route such response data to the data building engine 110, e.g., to a corresponding virtual network interface, for processing of the stateful response data accordingly.

[0041] Figure 4 An example of logic that a system can implement to support data mixing of management data and synthetic traffic data for validation of a DUT is shown. For example, the computing system 102 can implement the logic 400 as hardware, executable instructions stored on a machine-readable medium, or a combination of both. The computing system 102 can implement the logic 400 via the data building engine 110 and the data mixing engine 112, which the computing system 102 can perform or execute as a method to support data mixing of management data and synthetic traffic data for validation of a DUT. The following description of the logic 400 is provided using the data building engine 110 and the data mixing engine 112 as examples. However, other various implementations of the system are also possible.

[0042] In implementing the logic 400, the data build engine 110 can build management data to configure a DUT in a given manner (402). The DUT can take the form of a network device or network communication system, the DUT can be represented by a verification system, and the management data can take the form of any data by which the DUT (and thus the network device) can be configured to operate in a particular state according to a protocol or in any manner described herein. In implementing the logic 400, the data mix engine 112 can access synthetic traffic data to test operation of the DUT (404), such as synthetic traffic data generated locally on the computing system 102.

[0043] In implementing the logic 400, the data mix engine 112 can also mix the management data and the synthetic traffic data into a mixed data stream (406), and transmit the mixed data stream including the management data and the synthetic traffic data to a verification platform representing the DUT over a physical communication link (408). As used herein, “communication of the mixed data stream” can refer to the actual transmission of the mixed data stream over the physical communication link, or to any operation that supports the transmission of the mixed data stream over the physical communication link. Thus, the data mix engine 112 can transmit the mixed data stream by transmitting data packets over the physical communication link itself, by passing the mixed data stream to an interface (e.g., a software interface or a network interface) of the computing system 102 for a next step in the communication process, or by any other suitable operation that supports the transmission of the mixed data stream.

[0044] Figure 4 The illustrated logic 400 provides an illustrative example by which the computing system 102 can support, implement, or provide data mixing functionality for management data and synthetic traffic data. Additional or alternative steps in the logic 400 are contemplated herein, including in connection with the data build engine 110, the data mix engine 112, or a combination of both, according to any data mixing techniques described herein.

[0045] Figure 5 An example of a computing system that supports data mixing of management data and synthetic traffic data for verification of a DUT is shown. The computing system 500 can include a processor 510, which can take the form of a single or multiple processors. The one or more processors 510 can include a central processing unit (CPU), a microprocessor, or any suitable hardware device for executing instructions stored on a machine-readable medium. The computing system 500 can include a machine-readable medium 520. The machine-readable medium 520 can take the form of any non-transitory electronic, magnetic, optical, or other physical storage device used to store executable instructions such as Figure 5The illustrated data build instructions 522 and data mix instructions 524. Thus, the machine-readable medium 520 can be, for example, random access memory (RAM) such as dynamic RAM (DRAM), flash memory, spin-transfer torque memory, electrically erasable programmable read-only memory (EEPROM), a storage drive, an optical disc, and the like.

[0046] The computing system 500 can execute instructions stored on the machine-readable medium 520 by the processor 510. Execution of these instructions, e.g., the data build instructions 522 and / or the data mix instructions 524, can cause the computing system 500 to perform any of the data mixing functions described herein, including according to any of the functions of the data build engine 110, the data mix engine 112, or a combination of the two.

[0047] For example, execution of the data build instructions 522 by the processor 510 can cause the computing system 500 to build management data to configure a DUT in a given manner. The DUT can take the form of a network device, the DUT can be represented by a verification system, and the management data can take any form such that the DUT can be configured to operate in a particular state according to a protocol or any of the manners described herein. Execution of the data mix instructions 524 by the processor 510 can cause the computing system 500 to access synthetic traffic data to test operation of the DUT, e.g., synthetic traffic data generated locally on the computing system 500.

[0048] Execution of the data mix instructions 524 by the processor 510 can also cause the computing system 500 to mix the management data and the synthetic traffic data into a mixed data stream, and transmit the mixed data stream including the management data and the synthetic traffic data over a physical communication link to a verification platform representing the DUT. As described herein, communication of the mixed data stream can refer to actual transmission of the mixed data stream over the physical communication link, or to any operations that support transmission of the mixed data stream over the physical communication link.

[0049] Any additional or alternative data mixing functions described herein can be implemented via the data build instructions 522, the data mix instructions 524, or a combination of the two.

[0050] The systems, methods, devices, and logic described above, including the data build engine 110 and the data blending engine 112, can be implemented in a number of different fashions, with various combinations of hardware, logic, circuitry, and executable instructions stored on machine-readable media. For example, the data build engine 110, the data blending engine 112, or a combination thereof, can include circuitry in a controller, microprocessor, or application specific integrated circuit (ASIC), or can be implemented with discrete logic or components, or other types of analog or digital circuitry, which can be combined on a single integrated circuit or distributed across multiple integrated circuits. Products such as computer program products can include storage media and machine-readable instructions stored on the media that, when executed in a terminal, computer system, or other device, cause the device to operate as described above, including according to any of the functionality of the data build engine 110, the data blending engine 112, or a combination thereof.

[0051] The processing capabilities of the systems, devices, and engines described herein, including the data build engine 110 and the data blending engine 112, can be distributed among multiple system components, such as multiple processors and memories, optionally including multiple distributed processing systems or cloud / network elements. Parameters, databases, and other data structures can be stored and managed separately, combined into a single processor or database, organized logically and physically in a number of different ways, and implemented in a number of ways, including data structures such as linked lists, hash tables, or implicit storage mechanisms. Programs can be part of a single program (e.g., a subroutine) or separate programs, distributed across multiple memories and processors, or implemented in a number of different ways, such as in a library (e.g., a shared library).

[0052] While various examples have been described above, there are many further modifications possible.

Claims

1. A method comprising: The operation of the DUT (122) is verified by a computing system (102) communicatively linked to a verification platform (120) representing the design under test (DUT) (122): Build (402) management data (210) to configure the DUT (122) in a given manner; Access (404) synthetic traffic data (220) to test the operation of the DUT (122); The management data (210) and the synthetic traffic data (220) are mixed (406) into a mixed data stream (230). as well as The mixed data stream (230), which includes the management data (210) and the synthetic data (220), is transmitted (408) to the verification platform (120) representing the DUT (122) via a physical communication link.

2. The method according to claim 1, wherein, The hybrid data stream (230) includes a data packet stream (310) transmitted from the computing system (102) to the verification platform (120) via the physical communication link, wherein mixing the management data (210) and the synthetic traffic data (220) into the hybrid data stream (230) includes inserting the management data (210) into the data packet stream (310) during an idle period of the synthetic traffic data (220) transmission.

3. The method according to claim 1 or 2, comprising: The management data (210) is constructed based on user input to configure the DUT (122) for verification in the given manner.

4. The method according to any one of claims 1 to 3, comprising: After the first portion of the synthesized traffic data (220) has been transmitted to the verification platform (120), the management data (210) is constructed, and This includes: mixing the management data (210) with another portion of the synthetic traffic data (220), the other portion of which is different from the first portion of the synthetic traffic data (220) that has been transmitted to the verification platform (120).

5. The method according to any one of claims 1 to 4, wherein, The management data (210) configures the status of the DUT (122), the communication protocol used by the DUT (122), or a combination of both.

6. The method according to any one of claims 1 to 5, wherein, The management data (210) includes data for the management plane of the DUT (122), and The synthetic flow data (220) includes data from the data plane of the DUT (122), data from the control plane of the DUT (122), or a combination of the two.

7. The method according to any one of claims 1 to 6, further comprising: The operation of the DUT (122) is verified by means of the verification platform (120) by simulating or emulating the operation of the DUT (122) represented by the verification platform (120).

8. A system (100) comprising: The verification platform (120) is configured to represent the design under test (DUT) (122) to verify the operation of the DUT (122); as well as The computing system (102) includes: The data building engine (110) is configured to build management data (210) to configure the DUT (122) in a given manner; and The data mixing engine (112) is configured as follows: Access the synthetic traffic data (220) to test the operation of the DUT (122); The management data (210) and the synthetic traffic data (220) are mixed into a mixed data stream (230); and The hybrid data stream (230), including the management data (210) and the synthetic data, is transmitted to the verification platform (120) representing the DUT (122) via a physical communication link.

9. The system (100) according to claim 8, wherein, The hybrid data stream (230) includes a data packet stream (310) transmitted from the computing system (102) to the verification platform (120) via the physical communication link, and The data mixing engine (112) is configured to mix the management data (210) and the synthetic traffic data (220) into the mixed data stream (230) by inserting the management data (210) into the data packet stream (310) during the idle period of the transmission of the synthetic traffic data (220).

10. The system (100) according to claim 8 or 9, wherein, The data building engine (110) is configured to build the management data (210) based on user input to configure the DUT (122) for verification in the given manner.

11. The system (100) according to any one of claims 8 to 10, wherein, The data construction engine (110) is configured to construct the management data (210) after the first portion of the synthesized traffic data (220) has been transmitted to the verification platform (120) by the data mixing engine (112), and The data mixing engine (112) is configured to mix the management data (210) with another part of the synthetic traffic data (220), the other part of the synthetic traffic data (220) being different from the first part of the synthetic traffic data (220) that has been transmitted to the verification platform (120).

12. The system (100) according to any one of claims 8 to 11, wherein, The management data (210) configures the status of the DUT (122), the communication protocol used by the DUT (122), or a combination of both.

13. The system (100) according to any one of claims 8 to 12, wherein, The management data (210) includes data for the management plane of the DUT (122), and The synthetic flow data (220) includes data from the data plane of the DUT (122), data from the control plane of the DUT (122), or a combination of the two.

14. The system (100) according to any one of claims 8 to 13, wherein, The verification platform (120) is also configured to verify the operation of the DUT (122) by simulating or mimicking the operation of the DUT (122) represented by the verification platform (120).

15. A non-transient computer-readable medium comprising instructions that, when executed by a processor, cause a computing system (102) to perform the method according to any one of claims 1 to 7.

Images